CircleOS Gestalt SDK – an ORM for Frontend Applications

Context

CircleOS makes you build applications faster through embedding them in a fully featured environment. In our previous post we were able to give an overview of the architecture behind it.

Let’s take another look.

As you can see, with CircleOS custom applications are installed into an environment that already provides routing, authorization and authentication as well as a file system that we can use as a database.

Now that we have delivered such a well-formed environment for new apps, we want to take it all to the next level: Let's transform the way an app is developed by keeping the following points in mind:

• We still want to develop, not configure an app.
• We want to keep full control, no restrictions.
• We want to be able to reuse functions that repeat over a set of applications.
• We would like to reuse UI components.
• We would love to write code that is easy to read and doesn't contain unnecessary complexity.
• We want to support tools and software languages that are well-known.
• We should have the ability to switch between coding and a dedicated WYSIWYG Editor for non-developers in the future.

As a result of these claims we created an SDK that:

• is based on Javascript, especially compatible with React.js.
• contains a library for visual components such as text inputs or buttons.
• contains a ORM library that handles common data operations.
• contains common utilities like date handling or requesting resources.

In this article we will talk about the ORM library that is one puzzle piece of our SDK to make app development with CircleOS a fun experience.

The typical ORMs - a recap

The term ORM means „object-relational mapping“ which is used for converting data between different type systems. The idea of an ORM is to save you a lot of trouble. Usually, a backend developer needs to understand a query language like SQL and is forced to use it inside the backend code. By using SQL, data from a database is queried and inserted. An ORM basically hides database queries by providing an easier to use interface so that the user doesn't have to be an SQL expert. It's a common abstraction for many object oriented languages.

Instead of writing SQL statements, a backend developer writes code which is translated into those queries.

Without an ORM, a lot of string concatenations and other nasty bits of code will be added. Resulting in something like this:

import { query } from "some-sql-lib";
import Entry from "./types/Entry";

async getEntries(filters) {
    // of course a lot of stuff happens until we can securely inject "filters"
    const result = await query(`SELECT * FROM entries WHERE ${filters}`);

    return result.map(x => new Entry(x));
}

async createEntry(entry) {
    // and also here we will need to secure entry.name and entry.speed
    const result = await query(`INSERT INTO entries ("name", "speed") VALUES ('${entry.name}', ${entry.speed})`);
    
    return new Entry({
        id:    result.body.id,
        name:  entry.name,
        speed: entry.speed
    });
}

As you can see, this is not scalable and the code is extremely bloated. Utilizing an ORM gets rid of these issues. Now, instead of writing SQL queries, you will merely have to describe the data and execute operations like create or update. Looking at our example code, this turns it into something that is much better to read and maintain:

import { getEntities, String, Integer } from "some-orm-lib";

// Some "Type" definition
const Entry = {
    name:  String,
    speed: Integer
};

getEntries() {
    return getEntities("entries");
}

createEntry(entry) {
    return persist(entry);
}

Very beautiful. But wait a minute... ORMs only run on backend side. Meaning a frontend developer needs to send requests against an API and map the response back into some data type. Therefore, the ORM effect is missing on frontend side.

Rethinking what we already know

At Circle, we asked ourselves: Why don’t we just take the idea of ORMs and connect the whole: A full stack ORM, so to speak, that handles the full transaction from frontend to database and back.

Of course this might be dangerous and insecure if done improperly. That's why we tackled it – the challenge is exciting.

The design

Before we look at how the ORM is used in the code, let's look at what happens in the background and how it works. While this knowledge is not necessary in order to use the system, it does help in making sense of what we have in mind.

Basically, as described above, we need to model the transaction from the frontend all the way to the database and back. Technically speaking, we need the following building blocks for this:

• A frontend
• A backend
• A database

However, since our goal is to map the entire process with an entity description and minimal business logic, all three parts must be reduced to their absolute minimum:

• The frontend must trigger operations on the data – whether it be to create, read, edit or delete. To do so it needs to be aware of the entity definitions.
• The backend has to automatically translate the operations that the frontend sends into SQL queries and, thus, also needs the entity definitions. It needs to provide endpoints for creating, reading, editing, and deleting and translate them into SQL.
• At the same time, the database needs the table definitions of the entities in order for the SQL queries to work.

As a result:

• The frontend needs to be programmed.
• The backend can be configured.
• The database can be configured.

Since the entity definition can be translated both into backend logic as well as into table definitions, all that is needed as a building block for these elements is the definition. More precisely, the entity definition configures the backend and data base. What still needs to be programmed is the frontend.

To make this work, the backend must be able to translate the entity definition into business logic – and that’s what CircleOS provides in the form of a docker image: a generic, entity-configurable backend that generates the table definitions and uses them to configure, read and describe the database.

The frontend, which has to be developed by the user, now controls the backend’s input and output. With the help of operations on the entities, messages are generated in the background, which are sent to the backend: reading messages or writing messages. To ensure that all frontend instances update in the same way, they are permanently connected to the generic backend via WebSocket and receive messages from other participants. This creates a quasi live system in which everything is synchronized with all other instances at all times.

The messages conform to a fixed definition and all follow the same structure, hence are standardized. This makes it possible to create arbitrary software development kits for different languages, which would transform the code into the communication scheme.

Usage

Let's see how developers use the tooling we provide for Single Page Applications based on Javascript.

Developers run an initializer script to get the latest state of the ORM. Additionally, a local or online environment will be spawned, so that the database and the generic server run. Next, the developers create an entity definition and the business logic.

This is how the entry entity definition looks like.

import { Table, varchar, integer, primary, serial } from "@circle/gestalt-app";

export default Table({
    id:    primary(serial)
    name:  varchar(60),
    speed: integer
}, "entries");

The business logic is also quite simple to code.

import { App as BasicApp, sync } from "@circle/gestalt-app";

// That's the frontend state
class App extends BasicApp {
    
    // makes sure that all operations work through ORM approach
    @sync
    entries = [];

    createEntry(entry) {
        return this.select("entries").push(entry);
    }

    getEntries() {
        return this.get("entries");
    }
}

If you want to make use of data, you can make your components subscribe to resources with the same principle.

import { branch } from "@circle/gestalt-app";

const EntriesOverview = props => {
    return (
        <div className="entries-container">
        {
            props.entries.map(entry => <p>{entry.name} | {entry.speed}</p>)
        }
        </div>
    );
}

export default branch({
    entries: ["entries"]
})(EntriesOverview);

All entries will be shown in the EntryOverview component as soon as they are created or modified. The "branch" function achieves that. It listens to the entries entity. Now, if your app creates an entry it will be inserted into the database through the generic backend. All actors connected to the backend – frontend services as well as backend services – get informed about the change and update their own state accordingly. All actors in the system are synchronized with each other.

The newly created entry doesn't only show up in the creator’s frontend, but all other frontends are also notified and show the new entry. The result: live collaboration as native principle.

Additionally, so to say as a byproduct, we get a fully working REST API for 3rd party services. Calling GET /entries will also return all existing entries.

The server checks authorization and authentication as well, so that no one gets access without privilege. The security mechanism is not described within this post, but might be a topic to discuss on our blog in the future.

CircleOS' ORM, which we call "Gestalt", changed everything for us. Traditional web applications no longer need a backend because the full stack ORM abstracts away the backend like a backend ORM abstracts away the database.

The only remaining reason for backend code is specific backend logic that you want to hide from the frontend. Now that we have a pub/sub broker, the backend services also subscribe to resources and push data, resulting in a beautiful, stream-oriented architecture for all kinds of services.

Benefits

So what are the benefits of Gestalt within CircleOS?

• No backend coding for simple CRUD operations (low-code architecture)
• Live collaboration between multiple clients
• Less code for common frontend applications
• Developing a web SPA application can be learnt more easily

Our customers love the collaborative behavior of all the apps, because it feels like you run native applications inside a browser thanks to Gestalt.

If you want to take an exclusive look at it today, get in touch with us.

And if you want to help us build this future and drive other important outcomes, hit us up 😉.