OrgPage screenshots

Created by Pavel Klavík

#OrgPad

Fix transparent just-image printing

Progress for printing into PDF

New Chrome task retrieved

Add it to the queue

If the number of running tasks is less than MAX, move the top task to running and run it immediately

When done, remove from running tasks

Use fragment images when displaying fragments in OrgPage embeds

Create screenshots for all fragments

They will be accessible at routes /o/.../screenshot?l=fragment and similar. l=fragment will be either text-id or fragment-id.

Improve timing estimates

Check the queue again

Screenshot server improvements

State stored as screenshot-db.

Running tasks

Queued tasks

With two priorities.

Monitoring sent to the main server every 10 seconds

As a single Clojure map.

Monitoring records

Numbers of running and queued tasks per period

Number of finished tasks per period

Durations of finished tasks within the period

• time, when added to the queue
• time, when started running the task
• time, when the task is finished

environment.txt

We need to add a few variables:

• SCREENSHOT_PORT=3100
• CONF=config.edn

Running npm install there

It will update all new dependencies. We are not changing them often, so it should be finished immediately.

Running the server

node screenshot.js

We need to add this as a service, so we can do systemctl screenshot start, etc. Environment variables need to be shared.

Building

It is done as part of uberjar compilation:

shadow-cljs release screenshot

Generates screenshot/screenshot.js.

Copying to some directory on the server

Together with package.json and config.edn in screenshot directory.

Deployment

Block all operations

So we don't see changes in the screenshot when someone is still working on the Orgpage.

Generating thumbnails

Should be relatively easy using this library:

https://sharp.pixelplumbing.com/

Set exact zoom level

Instead of rounding it, so we see as much as possible.

Display only Orgpage for screenshot

When screenshot=true is set, remove panel and all popups.

Also, add #screenshot into DOM when Orgpage is loaded.

Example code in ClojureScript

Without Shadow-cljs, there it should be even simpler.

https://gist.github.com/apeckham/ec1725b22642d9344e294c701de18fb2

Request for Orgpage route adds image url for FB, etc.

Assuming that the Orgpage is public or a valid token is passed within the URL.

Storing images directly to S3, also their thumbnails

https://stackoverflow.com/questions/56693877/what-is-the-recommended-way-to-save-image-data-from-a-puppeteer-screenshot

Puppeteer

https://github.com/puppeteer/puppeteer

Orgpad client

Blog post with example code

https://bitsofco.de/using-a-headless-browser-to-capture-page-screenshots/

OrgPad server

Running HTTP server on some port

We want to run it on localhost:3100. It should be accessible only from local machine (i.e., OrgPad server), not from outside. (At least for this moment.)

Remove sending preview sizes

Request for a preview or its thumbnail redirect to S3

After checking that you can access this resource.

Remove preview sizes

NodeJS server

Maybe we can build it with this: https://macchiato-framework.github.io/. We don't really need much from it anyway.

Schedules regeneration of screenshots

After every change, the screenshot is generated in 5 minutes. Further changes do not postpone this.

Scheduler sends an HTTP request

Containing orgpage-id and read access token.