Background:⌗

wikipedia

r/place is a recurring collaborative project and social experiment hosted on the content aggregator site Reddit. The 2017 experiment involved an online canvas located at a subreddit called r/place. Registered users could edit the canvas by changing the color of a single pixel with a replacement from a 16-color palette. After each pixel was placed, a timer prevented the user from placing any more pixels for a period of time varying from 5 minutes.

Part 1 - getting the canvas images.⌗

2023 event started and I didnt plan to be a part of it, but as I made a few timelapses last year some people asked if I could create new ones.
I didn’t find a datadump of the recorded images (yet) so I looked to place-atlas to see if I could find their source.

Didnt find it, so decided to see if I could data dump their images with a hacky python script.
Started with inspecting their site trough firefox + inspect + network and moving the slider to see what popped up.
It seemed like their time-slider was loading images with javascript, not staticly declared in the source code.. Though the filenames were the epoch-time of the timestamp tooltip!
The slides had URL-hooks, numbered 0-260, that’s possible to iterate.

from selenium import webdriver
from bs4 import BeautifulSoup
from datetime import datetime
import time
import calendar
import requests
import shutil

logfile = open("skipped", "w") # file to log skipped slides

# # Uncomment to retry with list of skipped slides, comment other 'for'-start:
# skipped = open("skipped", "r")
# for number in skipped.readlines(): # retrying

for number in range(1,259): # iterate over the slides, skipping first and last (blank)
    try:
        driver = webdriver.Firefox()
        print("Current slide:{}".format(number))

        url = "https://2023.place-atlas.stefanocoding.me/#/{}".format(number)
        driver.get(url)
        time.sleep(3) # sleep to properly load the page and JS-code

        html = driver.page_source
        soup = BeautifulSoup(html, 'html.parser')
        
        # find the timestamp from slider tooltip
        timestamp = soup.find("div", {"class": "bg-body p-1 rounded"})

        driver.quit()

        # Get url for source image by converting timestamp to unixtimeUTC:
        img_date = datetime.strptime(timestamp.text, "%a, %d %b %Y %H:%M:%S %Z")
        img_unixtime = calendar.timegm(img_date.utctimetuple())
        img_url = "https://2023.place-atlas.stefanocoding.me/_img/canvas/main/{}.png".format(img_unixtime)

        # Grab the image and save it
        filename = "{}_atlas.png".format(number)
        res = requests.get(img_url, stream = True)
        if res.status_code == 200:
            with open(filename,'wb') as f:
                shutil.copyfileobj(res.raw, f)
    except:
        # logging skipped slides if it times out or any other error, to try again later
        print("Skipping slide {}".format(number))
        logfile.write("{}\n".format(number)) # log skipped number
        continue
logfile.close()

That’s the first step of getting raw image data to work with. Stealing them from place-atlas.

Part 2 - Processing the raw data⌗

After the first dump, I started working on a script to create timelapses from the raw images (I’ll continue on that in part 3).

Still searched for the full raw data and messaged u/TCOOfficiall who linked me to an archive. I downloaded the full collection of raw images which luckily had a logic naming scheme.

The full canvas were split in 6 parts, like this:

0 1 2
3 4 5

And on the actual canvas - like this: Canvas Splits

And files were named epochtime-area.png - example:

1689903000-0.png
1689903000-1.png
1689903000-2.png
1689903000-3.png
1689903000-4.png
1689903000-5.png

So that way I could just combine all 6 splits to one single canvas with this:

montage 1689903000-0.png 1689903000-1.png 1689903000-2.png 1689903000-3.png 1689903000-4.png 1689903000-5.png -geometry +0+0  montage1.png

# Worked! Next I needed to create a list of all timestamps to iterate over:
ls -1 *.png | sed 's/......$//' | uniq > list
  • list all the png’s, one line per file
  • remove the trailing -0.png,
  • remove the duplicate lines
  • write it to the file list.
  • Note:
    • the “images2” directory has reverse naming 0-1689903000.png
    • change to this: ls -1 *.png | sed -e 's/^..//;s/....$//' | uniq > list

Iteration!⌗

Loop over every timestamp(filename) to create a single canvas per timestamp:

while read -r line
do
  montage ${line}-0.png ${line}-1.png ${line}-2.png ${line}-3.png ${line}-4.png ${line}-5.png -geometry +0+0 montage_${line}.png
done < list

Though the canvas area grew over time from the center outwards, so prior to the start of the last period the outer splits were just not created.

This is how the canvas grew - and as you can see, it’s not within the same boundaries as the splits above. Canvas Growth

So I’d have 1689903000-1 1689903000-2 1689903000-3 1689903000-4 and missing 0 and 5.
This lead to trouble when merging all images of every timestamp to a single file, as I needed the “empty” areas filled to keep aspect ratio constant.

Next step - create a filler⌗

..to fill where files are missing (due to empty splits).

while read -r line
do
  [[ -f ./${line}-0.png ]] && canvas_0="./${line}-0.png" || canvas_0="xc:white"
  [[ -f ./${line}-1.png ]] && canvas_1="./${line}-1.png" || canvas_1="xc:white"
  [[ -f ./${line}-2.png ]] && canvas_2="./${line}-2.png" || canvas_2="xc:white"
  [[ -f ./${line}-3.png ]] && canvas_3="./${line}-3.png" || canvas_3="xc:white"
  [[ -f ./${line}-4.png ]] && canvas_4="./${line}-4.png" || canvas_4="xc:white"
  [[ -f ./${line}-5.png ]] && canvas_5="./${line}-5.png" || canvas_5="xc:white"
  montage ${canvas_0} ${canvas_1} ${canvas_2} ${canvas_3} ${canvas_4} ${canvas_5} -geometry 1000x1000+0+0 ./MONTAGE/${line}.png
done < list
  • [[ -f ./${line}-0.png ]] checks if the file exists.
  • If it’s there, sets the variable canvas_0 to that file, otherwise to "xc:white"
  • xc:white just fills the space with blank white.

Final montage loop - cleaned it up a little.⌗ 

while read -r line
do
  for num in {0..5}
  do
    [[ -f ./${num}-${line}.png ]] && declare canvas_${num}="./${num}-${line}.png" || declare canvas_${num}="xc:white"
  done
  montage ${canvas_0} ${canvas_1} ${canvas_2} ${canvas_3} ${canvas_4} ${canvas_5} -geometry 1000x1000+0+0 ./MONTAGE/${line}.png
done < list
  • for every line in the list, loop it over 0-5 numbers.
  • check if filename exist (epoch-timestamp+number)
  • if the file is missing, set that canvas split to fill blank white.
  • stitch together all 6 pieces.

Part 3 - turning raw images into a timelapse⌗

Now we’ve prepared all the images needed to create a timelapse of the whole canvas.

Creating the cropped timelapses!⌗

Anyone can create a full timelapse from images with ffmpeg, it’s just a oneliner like this (framerate is based on 8622 files / 60 seconds):

ffmpeg -framerate 144 -pattern_type glob -i '*.png' -c:v libx264 -pix_fmt yuv420p -filter:v fps=fps=60 FULL.mp4

Instead my niche was to create cropped, zoomed in timelapses of different artworks. The canvas is huge and different communities / subreddits claimed their piece and worked on it throughout the event.

But to easily communicate requests and pick the right areas to crop - I needed some tooling. The Atlas was already established - it was the perfect place to pick coordinates.

Calculating coordinates on the other hand.. At first I started doing it manually by calculating a crop over the area, scalable with 1920x1080.

That got tedious quickly and I created a script which over time grew to this:

Calculate the crop area and aspect ratio⌗ 

import os
import climage

# Function to sanitize the file/directory-naming
def cleanFilename(sourcestring, removestring=" %:/,.\\[]<>*?"):
    return ''.join([c for c in sourcestring if c not in removestring])

# Function to calculate 16:9 or 9:16 aspect ratio based on input
def image_to_ratio(w, h):
    User_Width= w
    User_Height= h

    while True:
        base = w//16 if w/h < 16/9 else h//9
        base_resolution = ((base * 16), (base * 9))
        # if the suggested resolution is smaller than the user defined, pad with 4 pixels and retry
        if base_resolution[0] < User_Width or base_resolution[1] < User_Height: 
            w+=4
            h+=4
        else:
            if User_Width > User_Height: # portrait or landscape
                return base * 16, base * 9, (1080 / (base * 9)) * 100
            else:
                return base * 9, base * 16, int((1080 / (base * 9)) * 100)

# User input:
name = cleanFilename(input("Name of Community/Artwork: "))
start = list(int(x.strip()) for x in input("Start coordinates: (Top left corner - example: -327,-186) ").split(','))
stop = list(int(x.strip()) for x in input("Stop coordinates: (Bottom right corner - example: 94,-74) ").split(','))

# Adjust for negative coordinates by the atlas: w-1500,h-1000
# The canvas is 3000x2000 and their coordinates is 0 in the center
# While the image processing is 0 in top left corner
start[0] += 1500
start[1] += 1000
stop[0] += 1500
stop[1] += 1000

# calculate resolution based on coordinates
user_resolution = ((stop[0]-start[0]), (stop[1]-start[1]))
print(f"UserRes: {user_resolution}")

# calculate 16:9 aspect ratio and scale percentage
ratio = image_to_ratio(user_resolution[0], user_resolution[1])
print(f"ClosestRes + Multiplier: {ratio}")

# new starting coordinates based on resize of cropped area
new_start = (start[0]-int((ratio[0]-user_resolution[0])/2), start[1]-int((ratio[1]-user_resolution[1])/2))

# create preview of the final image, with my template-canvas.png as base
os.system(f"\nconvert canvas.png -crop {ratio[0]}x{ratio[1]}+{new_start[0]}+{new_start[1]} -filter point -resize {ratio[2]}% test_{name}.png")
# print the line to use in the next step - timelapsing!
print(f"\nconvert canvas.png -crop {ratio[0]}x{ratio[1]}+{new_start[0]}+{new_start[1]} -filter point -resize {ratio[2]}%")

# climage - preview the image in the terminal:
preview = climage.convert(f"test_{name}.png", is_unicode=True, is_truecolor=True, is_256color=False, is_16color=False, is_8color=False, width=120, palette="default") 
print(preview)

With that done, I got a preview out of the final crop and the variables I need for the next script, the actual timelapse.

Time for timelapse!⌗

Now we have the values from the previous script to add to the Crop-variable.

Then I wanted to create a way of choosing start time, as many artworks were in areas not “unlocked” until later in the event. We’ve got the filenames as epoch-time still and people grab the coordinates from the Atlas - so the tooltip timestamp is what we’ll use.

#!/bin/bash
Crop='-crop 48x27+49+1595 -filter point -resize 4000.0%'
ProjectName="DFTBA"
Timestamp="Mon, 24 Jul 2023 02:30:00 GMT"
Length=30 # length of the output clip in seconds

Epoch=$(date -d "${Timestamp} -30 minutes" +"%s")

IFS=$'\n'
FileArray=( "$(ls -1 -- *.png)" ) # adding all png's to array
FileArray+=("${Epoch}.png") # adding fake timestamp-file to array
FileArray=("$(sort <<<"${FileArray[*]}")") # sorting
unset IFS
# remove everything before timestamp file, including the timestamp-file
CleanArray=$(echo "${FileArray[@]}" | sed -ne '/'"$Epoch"'/,${/'"$Epoch"'/!p;}')

# count frames to calculate length
ImageCount=$(printf '%s\n' "${CleanArray[@]}" | wc -l) 
Frames=$(( ImageCount / Length ))

mkdir -p "$ProjectName"

# iterate all the images in the array, crop+scale
Count=0
for f in $CleanArray ; do
  (( Count++ ))
  convert "$f" $Crop "$ProjectName"/"${f/.png/-cropscale.png}"
  # made a counter to show the process
  printf "Processed %s of %s\033[0K\r" "${Count}" "${ImageCount}"
done

cd "$ProjectName" || exit # quit if there's no directory
# stitch together the clip from the cropped images
ffmpeg -framerate $Frames -pattern_type glob -i '*.png' -c:v libx264 -pix_fmt yuv420p -filter:v fps=fps=60 $ProjectName.mp4

cd .. # go back to previous directory before quitting
exit

So that’s that! A timelapse is created.

  • Pick coordinates from the Atlas
  • Run through a python script to calculate the crop ratio and resolution
  • Run through a bash script to mass-crop the images and finally create the timelapse

Cant we do all that in one go?⌗

Well yeah, and over time I kept adding functionality to it so now its rather big considering where we started..

Terminal screenshot of lapsing the Ukraine-flag, coordinates picked from the Atlas:
Screenshot Ukraine

Still got some cleaning to do.. but at the time of writing - this is it:

import sys
import glob
import os
from datetime import datetime
import calendar
import climage

raw_images = "ALL/" # location of raw images
canvas_temp = "canvas.png" # template canvas file
clip_length=30 # length of the output clip in seconds

def clean_dirname(sourcestring, removestring=" %:/,.\\[]<>*?"):
    return ''.join([c for c in sourcestring if c not in removestring])

def image_to_ratio(w, h):
    user_width= w
    user_height= h

    while True:
        base = w//16 if w/h < 16/9 else h//9
        base_resolution = ((base * 16), (base * 9))
        # if the suggested resolution is smaller than the user defined, pad with 4 pixels and retry
        if base_resolution[0] < user_width or base_resolution[1] < user_height: 
            w+=4
            h+=4
        else:
            if user_width > user_height: # portrait or landscape
                return base * 16, base * 9, (1080 / (base * 9)) * 100
            else:
                return base * 9, base * 16, int((1080 / (base * 9)) * 100)

def cd(dir):
    if os.path.exists(dir):
        os.chdir(dir)
    else:
        print(f"{dir} does not exist, exiting.")
        exit()

def uniq_dir(dir):
    counter = 1
    dir_name = dir

    while os.path.exists(dir):
        dir = f"{dir_name}_{counter}"
        counter += 1

    return dir

def progress_counter(current, total):
    percentage = round(100.0 * current/float(total),1)
    sys.stdout.write(f"{current}/{total} images processed, {percentage}% complete \r")
    sys.stdout.flush()


# User input:
project_name = clean_dirname(input("Name of Community/Artwork: "))
start = list(int(x.strip()) for x in input("Start coordinates (format -123,456) :").split(','))
stop = list(int(x.strip()) for x in input("Stop coordinates (format -456,789) :").split(','))
timestamp = input("Atlas Timestamp for starting (leave blank for default): ") or "Thu, 20 Jul 2023 13:01:20 GMT"

# Adjust for negative coordinates by 2023.place-atlas.stefanocoding.me , w-1500,h-1000
start[0] += 1500
start[1] += 1000
stop[0] += 1500
stop[1] += 1000

# calculate resolution based o coordinates
user_resolution = ((stop[0]-start[0]), (stop[1]-start[1]))

# calculate 16:9 aspect ratio and scale percentage
ratio = image_to_ratio(user_resolution[0], user_resolution[1])

# new starting coordinates based on resize of cropped area
# Need to add logic for end of the canvas, in all directions
new_start = [start[0]-int((ratio[0]-user_resolution[0])/2), start[1]-int((ratio[1]-user_resolution[1])/2)]

# logic to stay within the canvas max resolution
for i, s in enumerate(new_start):
    new_start[i] = max(0, new_start[i])
if (new_start[0]+ratio[0]) > 3000:
    new_start[0] = new_start[0] - ((new_start[0]+ratio[0]) - 3000)
if (new_start[1]+ratio[1]) > 2000:
    new_start[1] = new_start[1] - ((new_start[1]+ratio[1]) - 2000)

# create preview-image with imagemagick
os.system(f"\nconvert {canvas_temp} -crop {ratio[0]}x{ratio[1]}+{new_start[0]}+{new_start[1]} -filter point -resize {ratio[2]}% test_{project_name}.png")

print("Preview of the cropped area, will look more pixelated due to the terminal:")
## Climage - show image:
preview = climage.convert(f"test_{project_name}.png", is_unicode=True, is_truecolor=True, is_256color=False, is_16color=False, is_8color=False, width=120, palette="default") 
print(preview)

# Continue? Long time!
user_choice = input("Looks good? Continue with the convertion (can take a while)? y/n: ")
if not (user_choice.lower() == "yes" or user_choice.lower() == "y") :
    exit()

crop_arguments = f"-crop {ratio[0]}x{ratio[1]}+{new_start[0]}+{new_start[1]} -filter point -resize {ratio[2]}%"

epoch = calendar.timegm(datetime.strptime(timestamp, "%a, %d %b %Y %H:%M:%S %Z").utctimetuple())

origin_dir = os.getcwd() # Get current dir for later
raw_images = raw_images.removesuffix("/") # sanitize input
cd(raw_images) # cd to the raw images

file_list = glob.glob("*.png") # list all pngs
file_list.append(f"{epoch}.png") # append fake timestamp-file
file_list.sort() # sort the list
# Grab only images after timestamp
fake_index = file_list.index(f"{epoch}.png") + 1
clean_list = file_list[fake_index:]

# count images to calculate length
image_count = len(clean_list)
frames = image_count / clip_length

# define name and create dir
work_dir = uniq_dir(project_name)
os.mkdir(work_dir)
print(f"Output directory: {work_dir}")

# iterate all the images in the list, crop+scale
que_number = 0
for file in clean_list: # TODO - add a break!
    que_number += 1
    new_filename = file.replace(".png", "-cropscale.png")
    convert_run = f"convert {file} {crop_arguments} {work_dir}/{new_filename}"
    os.system(convert_run)
    progress_counter(que_number, image_count)


cd(work_dir)
# stitch together the clip from the cropped images
ffmpeg_run = f"ffmpeg -loglevel error -stats -framerate {frames} -pattern_type glob -i '*.png' -c:v libx264 -pix_fmt yuv420p -filter:v fps=fps=60 {project_name}.mp4"
os.system(ffmpeg_run)
cd(origin_dir) # return to the starting directory

print(f"All done! Timelapse at {raw_images}/{work_dir}/{project_name}.mp4")
exit()

Watch the complete timelapse created from the previous screenshot.⌗

Now I’m working on a graphical “pick by mouse” feature and also to auto-pick timestamp based on coordinates.
To be continued..? Currently a work in progress at github: mag37/rplace_timelapse