THORN is a digital content productivity tool designed to help you sort through the fragments of your thinking, write more comfortably, and share your creations with your readers more easily.
The internal test was launched in November last year. Although THORN’s emphasis on interaction design and writing experience has been recognized by many users, many users complained that THORN’s experience was not good when the network environment was unstable. At the time, our first reaction was that it didn’t seem like a THORN problem.
But then, a question popped into our minds, is the Internet really necessary for writing applications? Taking this as an opportunity, the team’s ensuing discussions extended a range of issues, particularly regarding privacy and security.
We decisively paused the development work and stopped to rethink what kind of software application THORN should be? Then, based on the foundations of privacy, speed, and practicality, after a lot of research and practice, we began to recognize the limitless possibilities of local priority over THORN:
We believe that data ownership and real-time collaboration are not mutually exclusive. Software can be created with all the benefits of cloud applications, while also allowing you to retain ownership and control over all the data you create.
We refer to this type of software as local-first software because it prioritizes local storage over servers in remote data centers.
In cloud applications, the data on the server is considered the primary copy of the data; if the client has a copy of the data, it is only slave to the server’s cache. Any data modifications must be sent to the server. In a local-first application, we swap these roles: we treat the copy of the data on the local device (laptop, tablet, or phone) as the primary copy. Servers still exist, but they keep secondary copies of data to facilitate access from multiple devices.
Immediately afterwards, we were enlightened, as if all the problems were solved.
Going back to the previous description of local-first software, it seems to be no different from some local writing software with cloud backup, right? Not really, please keep reading.
Not abandoning the cloud, but local first
For multi-device synchronization and user collaboration, traditional cloud-based solutions have been successful with many products and services. Although they allow you to access your data anywhere, all data access must go through their servers and you are only allowed to do what the server allows you to do. So all your data, including the history of changes to that data, is fully recorded.
In other words, you don’t fully own the data at all, the ownership and control of the data are in the hands of the service providers, you can’t resist, you can only think that these service providers will follow their proposed user agreement and privacy policy, and handle you properly of all data.
On the other hand, you are also at the mercy of the company that provides the service. If the service is unavailable or down, you will no longer be able to access the data created with the software, and even if you can export the data, in most cases you will not be able to use the software normally without a server. This is also the reason why most users trust large companies or companies with strong capital backgrounds. These companies are less likely to fail, so they can provide longer and more stable services.
You may have used software that syncs data between devices via iCloud or WebDAV, and the data is usually saved in files on the local disk, so you have full control and ownership of the data: you can do Anything you want, including long-term archiving, making backups, and manipulating files with other programs.
You don’t need anyone’s permission to access your files anytime, without going through a server run by a company. These software usually guarantee your absolute privacy without any censorship. However, they don’t bring you features like real-time synchronization, online collaboration, and more.
So between privacy and collaboration, you need to make a tough trade-off. Can’t we have the best of both worlds?
Of course, the answer is ” local first “. Local-first software has seven characteristics:
- Fast response : The primary copy of the data is kept on the local device, and the user never has to wait for a network connection. Data synchronization with other devices and users happens silently in the background.
- Multi-device synchronization : Data is kept in local storage on each device, and the data is automatically synced across all the devices the user works on.
- Network optional : Users can read and write data at any time, whether online or not. When a network connection is available, the native device automatically synchronizes with other devices.
- Collaboration : Native devices and other devices (whether those are yours or not) support real-time collaboration on the same data.
- Longevity : Your data should be accessible indefinitely. Since you have a native copy of the software and data, this software will work forever. Even if the software maker goes bankrupt, you can continue to run the last released version of the software. And you can export all of your data to a common format for access with other software.
- Security and privacy : Unlike traditional cloud-based solutions, local-first software does not have a centralized database that holds all of your user’s data, your local device only stores your own data, saving everyone by avoiding the use of a centralized cloud database data, local-first software has great security and privacy.
- Data ownership and control : Data ownership and control here is not in the legal sense, but refers to the local first software manufacturer will not restrict your access to the local copy data, you are allowed to copy and modify it by any means at any time This data without having to access the data through the service provider’s API.
Seeing this, everyone should have a sufficient understanding of local-first software. But unfortunately, the above description is only an ideal state.
So how exactly should we implement local-first software as much as possible?
Infrastructure for Local First Software: CRDT
According to the related research of Ink&Switch, CRDT is one of the basic technologies most likely to be used for local-first software implementation.
A CRDT is a special data structure that enables multiple devices to collaboratively edit the same data object. Specifically, if device A and device B are now simultaneously editing a data object data, over time, two change sequences, changes-on-A and changes-on-B, are generated, respectively, as long as device A and device B are respectively Send your changes to each other one by one, then the two devices can calculate the final state of the data locally.
Sounds like nothing special, right? But the magic is that CRDT is mathematically proven to ensure that as long as all changes to data are received from the other party (don’t care about the order in which the changes arrive), the final state of data calculated by the two devices must be consistent.
We can imagine that there is no central server to process and resolve data conflicts of all users, but each device resolves conflicts by itself, and CRDT can ensure that when each device calculates by itself, all devices can still calculate the complete consistent results.
The above description is not accurate, but it can already express the core advantage that CRDT can bring, decentralization .
The premise of decentralization is to apply and implement CRDT on the client side, because existing collaboration software can also use CRDT technology in its server software to resolve data conflicts. The more representative ones are Azure Cosmos DB , Redis , Riak , Weave Mesh , SoundCloud’s Roshi and Facebook’s OpenR .
If you’re still interested in continuing to explore CRDTs, check out this article by Alexei Baboulevitch ( Data Laced with History ) and this video by Martin Kleppmann ( CRDTs and the Quest for Distributed Consistency ).
At this point, I think you already understand what a CRDT means for local-first software.
We believe that CRDTs have the potential to be the foundation of a new generation of software. Just as packet switching is the enabling technology for the Internet and networking, or capacitive touchscreens are the enabling technology for smartphones, we think CRDTs could be the foundation of collaborative software that gives users full ownership of their data.
THORN’s data synchronization engine
Let’s go back to the specific implementation. You can notice that for CRDT, a server is still needed to ensure that when the client connects to the network, the server can know the data changes of other clients at any time from the server (of course, the client You also need to notify the server of changes to data when you are offline).
The above figure is a schematic diagram of the logical structure of THORN’s data synchronization mechanism. You can notice:
- All client devices of each user have a copy of the data stored in the local database;
- The THORN synchronization service also has a copy of the data, but it is stored in Alibaba Cloud OSS.
Strictly speaking, THORN’s official synchronization service is a more reliable “client” because it will save the encrypted data copy to Alibaba Cloud OSS with intra-city redundancy and remote disaster recovery enabled. Native storage media that can provide data persistence of up to 12 9s.
At the same time, when you actively delete a data object, the official synchronization service will broadcast the deletion event to all online devices, but if there is an offline device (due to network reasons and did not receive the broadcast deletion event), the deleted data will be It will be retained in this offline device (you can resync (restore) this data object at any time through this offline device later).
On the other hand, when any client device connects to the THORN sync service, the client and the THORN sync service will transmit data updates to each other to complete the data synchronization, and then the THORN sync service will push other clients’ data updates to the client At the same time, the data changes of the client will also be transmitted to the synchronization service at any time.
Therefore, there is a star architecture between the sync service and the client device:
As shown in the figure, the synchronization between clients 1, 3, 4, and the THORN synchronization service ensures that their respective data copies are in the same state.
When the client 3 reconnects to the network, it can also achieve the final consistency of the state of all data copies through data synchronization with the THORN synchronization service.
Seems like nothing special? Traditional cloud-based solutions are also similar to star architecture. This understanding is wrong. The traditional model is a centralized star architecture. The client is subject to the server. Without the server, the client can hardly run normally.
The THORN model is a star structure similar to the P2P network. The client and the server are in an equal relationship, and there is no synchronization service. The client can also run normally independently (but there is no user collaboration and multi-device synchronization).
The equality between the client and the server is reflected in that the server can be replaced. You can use the THORN official operation and maintenance synchronization service, or you can use the self-deployed THORN synchronization service. It is worth mentioning that the switching between synchronization services can be completed quickly. All you need to do is to change the synchronization service and resynchronize the full amount of data once.
Seeing this, I believe that there must be a big question in your mind. The aforementioned local-first software privacy and security does not seem to be guaranteed by this model of THORN?
How to ensure privacy and security?
Privacy and security issues actually only involve the THORN synchronization service itself, because under the aforementioned understanding, it does not belong to your assets and facilities, even if it exists to provide you with better services (multi-device synchronization and collaboration) ), to assist you in good faith, not to spy on you maliciously.
Aren’t you worried, some evil THORN team member, going to peek at your data? While you have your data locally, the sync service also has your data.
Maybe you don’t care about your data being snooped on by hackers, but in reality, many people simply cannot use cloud applications due to legal and regulatory constraints and confidentiality obligations.
So, how does THORN do it?
In THORN, each user can create multiple spaces, and each space can use different THORN synchronization services
It is very important to understand the above. Because we will implement different THORN synchronization services based on different technologies in the future. Different synchronization service implementations may have completely different degrees of privacy and security.
current plan
The current THORN synchronization service is implemented based on Websocket, which not only requires low computing and network resources, but also has high performance.
The THORN synchronization service operated and maintained by our official team uses Alibaba Cloud OSS with intra-city redundancy and remote disaster recovery enabled as the storage medium, but we will also support connecting to the object storage service purchased by users in the future. In addition, the THORN synchronization service will also support self-deployment by individuals and organizations.
It is mentioned that since the THORN synchronization service has no traditional database dependencies, even if individual users deploy the THORN synchronization service by themselves, the required resources and costs are extremely low.
Of course, this is all optional, depending on how you choose:
- Trust the THORN team : use the THORN official synchronization service to save a copy of the data to the THORN official object storage service;
- Maximum protection of data ownership and control : use THORN’s official synchronization service to save data to the object storage service purchased by yourself;
- Guarantee privacy and data sovereignty to the greatest extent : use the self-deployed official synchronization service to save data to the object storage service purchased by yourself;
1 In this case, when your data is disseminated, your data may be subject to review under the requirements of relevant laws and regulations. For details, please refer to the “Users’ Commitment to Legally Use THORN Services ” in the “THORN Terms of Service” part.
But no matter 1, 2 or 3, you should follow the “THORN Terms of Service” and use THORN legally. If you violate the laws and regulations of your region and the “THORN Terms of Service” , you will be prohibited from using THORN synchronization services.
future plans
Before mid-2023, we will launch a new Hypercore -based THORN synchronization service in the international version of THORN.
The Hypercore protocol is a P2P data network built on Hypercore logs. Hypercore is a signed, append-only log. They are like lightweight blockchains without consensus algorithms.
With Hypercore, THORN has end-to-end encryption.
Due to the completely different architecture, the data of the THORN internal beta version and the official version are not compatible, and cannot support migration, but internal beta users do not need to worry, we will provide a compensation plan.
The new official version of THORN
The cloud-based architecture solution of the THORN beta version is completely abandoned. The entire application architecture of the THORN official version supports “local first” natively at the bottom.
On the basis of “local first”, we have created a brand new THORN application.
Functional Architecture
The above picture shows the planned functions of the official version of THORN. For the specific function introduction, please visit the official website .
At present, all the functions are not completed, and the unfinished functions will be launched in subsequent updates.
Feature preview
Space overview
By clicking the button on the far left of the top bar, you can open the “Overview” to view the data statistics of the current space. At the same time, you can also create and switch workspaces on the overview page. In addition, the overview page also allows you to adjust the app display language and overview wallpaper. In the future, we will provide richer statistical visualization charts in the “Overview”.
Writing · All Articles
Writing · Full Text Search
Move the mouse over the search results list item and the context of the matching text inside the document will be displayed below.
Writing · Daily Notes
The most basic Snapshot function is now supported, and the integration of the Snapshot API with other applications will be supported in the future.
Writing · Article independent editing window
Writing · Text style menu
Text color/background, bold, italic, underline, strikethrough… it’s all there.
Writing · Node Menu
Paragraph Node Menu
The paragraph node integrates the Blockquote node and the Callout node, and is rendered in the new **marker** and **focus** paragraph styles, while also supporting paragraph **background** .
Enable image node with theme color background, title and rounded corners 
Image node menu 
Picture editing
The picture node supports dragging, pasting and inserting, supports picture title, rounded corners and automatic theme color background, and supports picture size and alignment style adjustment.
Formula Node Menu
Formula nodes support Latex and AsciiMath formula syntax, as well as chemical expressions.
Table Node Menu
The table node supports cell merging, background and alignment style adjustment, header style and drag-and-drop column custom width.
Mermaid graph node menu
Writing · Ordered/Unordered/To-Do List
Writing · Quick Command Menu
- Support fuzzy search
- Support Tab and up and down arrow keys to switch selected items
Writing · Online Web Pictures
Writing · Editor right panel
Article Index 
Article information 
Export and share
Writing · Exporting files
Export Markdown file 
Export PDF file 
export image file
This is just the beginning, there are many more amazing features and optimizations on the way…
write at the end
On August 1, 2022, THORN will be officially launched globally, and we look forward to seeing you soon!
The THORN early access version has been opened for download~
This article is reproduced from: https://sspai.com/post/74228
This site is for inclusion only, and the copyright belongs to the original author.