Created by OrgPad Info
Overview of used technologies in OrgPad and how they are linked together.
Garden is a Clojure library for describing CSS using Clojure data structures. It allows us to share certain parts of the code with the rest of the project and it is very convenient when you get used to it.
CSS description for OrgPad takes about 5000 lines of code where half of it is the list of all available icons for Font Awesome, so we include only used ones.
If you are interested, take a look at our OrgPage about Clojure.
We write our CSS files in Clojure.
user=> (css [:h1 [:a {:text-decoration "none"}]])
"h1 a{text-decoration:none}"
user=> (css [:h1 :h2 [:a {:text-decoration "none"}]])
"h1 a, h2 a{text-decoration:none}"The introductory website is fully built in OrgPad. So, one can immediately test it out and find out how OrgPad really works. We strongly believe in our product and want to showcase it. Further, any improvements we do for landing page also improve the rest of the application.
Volcano is a small Clojure library for developing static web in Clojure we have written and open sourced. You describe your website in terms of data. In development, it runs as a SPA in React with hot code reloading (changes in the code or CSS are at once shown in the browser). When you are done, you build small static HTML files which you can deploy.
All client data are stored in a single ClojureScript atom which is a deeply nested map. The main keys are:
:app-state:dashboard:orgpage:userWe generate email templates using Volcano in Clojure.
We send emails using Google Workspace SMTP relay.
Takes care of automatically improving how the units are placed. When a unit is opened, the surrounding units are pushed away to make a space for the content. This is important because otherwise you couldn't see its connections anymore. The computation runs a physics-inspired simulation where nearby units and links push each other away.
A web server placed in front of Orgpad server and Minio S3 compatible storage.
We use Sente on both client and server. Each message is a vector of the form
[:message-id] or [:message-id message-data],
where message-data is arbitrary Clojure data structure, often a map. We dispatch a multi function based on :message-id.
Subscriptions form a DAG of values computed from app-db and ending in components. Each subscription is a pure function of its inputs, so it has to be recomputed only when at least one of the inputs is changed. Therefore, a small change in app-db only leads to recomputing a few subscriptions and changing a few components.
Written in ClojureScript, using Shadow-cljs, React, Reagent and Re-frame.
 |  |  |
Events occur based on the user's interactions and the outside world (e.g., a message comes from the server). Each event produces a new app-db from the current one (and possibly some other values, i.e, cookies), and can have further effects on the outside world:
Written in Clojure using Ring and Netty. The code is relatively small and quite standard.
All data are stored in a PostgreSQL database, the total size more than 30 GB (October 2024) and ~ 8 GB if dumped and compressed using zstd -11. The database is growing quite a bit as new users join OrgPad. Most tables contain an arbitrary JSON field where we store data that does not need to be indexed. We use HugSQL for writing SQL functions retrieving/updating data in the database.
We create a backup every 30 minutes and replicate this backup every 4 hours to a different datacenter and then additionally to a different provider. See security OrgPad for more information.
We have been able to restore a complete deletion of an account. We also regularly test a full restore of the backup.
Since all the code runs in Clojure(Script), we describe all the routes in terms of a single Clojure data structure. Therefore, changing a route will change it in all parts of the application. We use the library Bidi for routing.
["/" {"index.html" :index
"articles/" {"index.html" :article-index
[":id" "/article.html"] :article}}])The look of buttons, panels, etc. is based on Material design from Google, namely the implementation Material-ui. We use Font Awesome for icons. We have a simple ClojureScript wrapper which allows us to specify components in terms of data:
[button/button #:button{:id :panel/share-orgpage
:tooltip "Share this OrgPage with others"
:icon :fa/share
:label "Share"
:type :button/bordered}]The rendering of the page is described in terms of components, in the form of hiccup:
These views are using subscriptions to display data from app-db.
For each route, we write a handler as a function taking the request map and returning the response map. A mutation often occurs, changing the state of DB, uploading to S3 storage, etc.
We control headless Chrome using Puppeteer. We open the Orgpage, wait till it is loaded and produce a screenshot. We also use it for printing to PDF.
We store uploaded files, images, videos etc. into Minio S3 compatible storage behind nginx. When the user is authorized to access these files by the application server, they are served using an internal redirect. We backup this data every 4 hours to a different datacenter.
We further use S3 storage at DigitalOcean for deduplicated and encrypted offsite backups using borg and rclone, this is done just after the first backup finished.
Written in ClojureScript using NodeJS. Very simple, just 600 lines of code. Runs on a dedicated machine. Produces screenshots of OrgPages which are used as previews and when sharing on social networks: example.
Our blog is displayed using React. All blog posts are stored in an atom on the server, so they are loaded in memory and quickly available.
Images and other resources are loaded from S3, they are placed in an OrgPage. Also cell contents from OrgPad can be directly inserted into blog posts, for example the list of LaTeX commands in math/chemistry was done in this way. We plan to do further improvements to the blog as we create new posts.
We use Stripe to process credit card payments for OrgPad. Payment information is entered directly in Stripe customer portal website, it takes care of renewals and we receive all updates using a webhook. We use Stripe Java SDK.
To speed-up access to database, we keep a lot of information immediately loaded in memory, in one large Clojure atom called cache-db. It includes information about all users, OrgPages, permissions. This allows to read information almost immediately, even for complicated queries such as retrieving a list of owned/shared OrgPages. And since reads are non-blocking, multiple threads can read at the same time. This led to a massive speed up in performance of our server, see average CPU load graph.
Before the server starts, we load everything from DB in parallel which takes about 15 seconds. Every change in DB is then also replicated to this atom. Currently, the size of cache-db is about 1 GB in memory and it grows little bit with opened documents.
In this blog post, we describe our struggles with browser rendering performance and how we improved the speed 100x.
Spring animations are amazing, they create the natural feeling within the app. This approach was popularized by designers in Apple but for whatever reason the rest of world (including the browser) is still stuck with sucky animations described by curves and durations.
Originally, we were using React Motion. For performance reasons, we have completely replaced it with our custom animation library consisting of about 5k lines of ClojureScript, orchestrating the detailed animations everywhere.
Math/chemistry can be used in cells, for example:
This is rendered in the screenshot server using MathJax library which produces the above SVG image for the code
(x+y)^n = \sum_{k=0}^n {n \choose k} x^k y^{n-k}.