Journey Platform: A low-code instrument for creating interactive person workflows | by Arjun Raman | The Airbnb Tech Weblog | Might, 2023

Journey Platform: Low-code notification workflow platform that permits technical and non-technical customers to create advanced workflows via a easy drag and drop person interface.

By: Arjun Raman, Ken Snyder, Mengting Li

Efficient communication hinges on delivering the proper message, to the proper viewers, on the proper time. At Airbnb, our purpose is to have interaction our customers — each company and hosts — by delivering inspirational and informational notifications via numerous channels, corresponding to e mail or in-app messages.

Traditionally at Airbnb, advanced notification workflows have been solely managed by engineering groups, with every workflow requiring the deployment of code. As our platform advanced, we acknowledged the necessity for a low-code or no-code resolution to streamline the creation of those intricate notification workflows. In response, the Advertising and marketing Know-how staff developed the Journey Platform, a robust instrument that allows non-technical customers to construct and ship customized notifications based mostly on our customers’ engagement with Airbnb.

The targets of the Journey Platform are:

1. Empower customers to simply create event-driven notification workflows utilizing an intuitive drag and drop interface.
2. Allow real-time execution of those notification workflows for well timed and related communication.
3. Supply a unified interface for managing transaction notifications, corresponding to upcoming journey reminders and promotional notifications.
4. Assure Service Stage Agreements (SLAs) for processing numerous varieties of notification workflows, together with transactional and promotional communications.
5. Cut back the time required to develop advanced notification workflows.

Journey Platform permits customers to iterate sooner by permitting self-serve workflow creation. It has decreased the time taken to help a brand new use-case from 1–2 months to simply 1–2 weeks.

The important thing parts of the Journey Platform are:

1. Journey Platform UI: WYSIWYG instrument permits customers to tug and drop parts and create a workflow. The workflow definition is then transformed to a customized DSL (Domain-specific language) which might be interpreted and executed by the workflow orchestrator.
2. Workflow Orchestrator: Mind of the system, the workflow orchestrator takes within the workflow definition DSL from the UI. As soon as a workflow is launched, it listens for occasions from the occasion retailer that may begin the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and depends on the Motion retailer to carry out particular duties.
3. Platform Retailer:

• Occasion Retailer: Pre-configured catalog of Kafka occasions which Journey Platform can take heed to and set off new executions of a workflow or move occasions to present workflow execution.
• Motion retailer: Repository of predefined, specific-purpose capabilities permits customers to carry out numerous duties, corresponding to sending emails, push notifications, or emitting Kafka occasions. Customized actions might be outlined and built-in into the instrument, making them accessible to all Journey Platform customers.
• Attribute retailer: Central repository for important knowledge, corresponding to person metadata (e.g. person’s geolocation, Airbnb search historical past, and so on.) and contextual data. It helps decision-making in workflow branching processes by exposing these knowledge as a parameter to set circumstances upon via the parameter supervisor.
• Customized shops: Means to create customized motion or attribute shops which aren’t already outlined within the platform.
• Workflow Orchestrator: Mind of the system, the workflow orchestrator takes within the workflow definition DSL from the UI. As soon as a workflow is launched, it listens for occasions from the occasion retailer that may begin the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and depends on the Motion retailer to carry out particular duties.

When crafting the UI for the workflow automation system, we aimed to create a well-recognized and intuitive expertise. Drawing inspiration from move charts, productiveness instruments with “inspector panels,” and incorporating drag and drop performance, we wished a platform the place customers may begin instantly with out consulting the guide.

We additionally had a purpose of utilizing progressive disclosure to incrementally allow the total depth of the platform capabilities, whereas protecting it easy for customers who solely want a small subset of the options. By utilizing wise defaults, and shifting extra advanced options into tabs and sub-screens, our superior customers may create distinctive options, going past the pre-planned use circumstances.

To edit the graph, we leveraged React Flow, an open-source library. This enabled us to show the graph, in addition to present fundamental operations like zooming, panning, shifting, and connecting nodes. On high of this basis, we added our customized node and edge parts, together with drag and drop performance for including new nodes and an inspector panel for modifying present ones.

To create the varieties displayed within the inspector panel, we applied a schema-based type system. This method offers a excessive degree of flexibility, permitting us to declaratively specify the UI for particular node enter/output fields as a part of their kind definitions. The system is inbuilt a type-safe method, making use of Thrift annotations and Java reflection. Based mostly on the schema data and UI-specific annotations, the interface shows the suitable type fields, assist textual content, and validation, guaranteeing our UI is routinely up-to-date with the platform’s capabilities.

Area-specific language

DSL offers a excessive diploma of flexibility and customization, permitting us to outline the construction and habits of the workflow. As an alternative of getting to hardcode a workflow within the workflow engine, we as an alternative have a generic workflow outlined that may execute any DSL-based workflow. Nodes and edges make up a workflow, with nodes representing particular person actions or duties and edges defining the dependencies and relationships between them.

The nodes and edges embrace all the mandatory data to outline a workflow corresponding to inputs, outputs, and parameters handed between nodes. The DSL generated by the UI is handed to the workflow orchestrator, the place the DSL parser executes it.

Journey Shops

The occasions, attributes, and actions shops are an integral a part of the backend, as they permit listening to occasions to begin workflow executions, filter customers, and execute duties within the journey. All these parts work collectively seamlessly to create a versatile and customizable backend that may be tailor-made to the precise wants of the platform.

Occasion Retailer

Journey Platform helps listening to totally different Kafka occasions and utilizing them to set off new executions of a workflow, or use the occasion to move alerts to a operating execution. For instance, begin a brand new execution when a visitor books a keep, move a sign to a operating execution when a person receives a push notification, and so on. Just like the motion retailer, as soon as an occasion is on-boarded, all of the groups at Airbnb can use it.

Attribute Retailer

The attribute retailer capabilities as a central repository for fetching all vital knowledge, corresponding to contextual knowledge, person preferences, and machine data, which can be utilized to complement the workflow branching course of and enhance decision-making capabilities. These shops are supported by an information storage system that manages numerous attributes or traits of entities.

Think about you’ve gotten a brand new person who simply signed up for Airbnb, and also you’re taken with figuring out whether or not they’ve carried out any itemizing searches on the platform. If the reply is sure, you’ll ship a customized message based mostly on their search historical past, and if it’s no, you’ll ship a static message.

This can be a concrete instance of how the Airbnb Journey Platform leverages attributes, corresponding to “itemizing search historical past,” to reinforce the person expertise. These attributes are extracted and outlined as parameters, which can be utilized for numerous functions. Every workflow execution has its personal parameter knowledge assortment, which might be accessed within the parameter supervisor. Extra details about parameters might be mentioned within the parameter supervisor.

Motion Retailer

The motion retailer is used to execute numerous duties, corresponding to sending an e mail or updating a database report, when a person reaches a selected level within the journey. It’s a frequent library the place every operate might be shared and reused by totally different customers of their workflow.

Every motion implements a typical interface, together with its metadata required for the UI schema-based varieties talked about above, and its habits in the course of the precise workflow execution.

Parameter Supervisor

Managing a fancy workflow that entails a number of steps with various inputs and outputs is usually a difficult process, particularly if the enter and output parameters change often or are totally different for every person. As an example, you would possibly want conditional branching in your workflow or customized communication content material based mostly on person search. That is the place parameterized workflows and parameter managers can show to be invaluable parts.

By specifying inputs and outputs (of attribute node / occasion node / customized node) as parameters, you’ll be able to reuse them all through your complete workflow execution. A parameter supervisor is a essential part that may retailer and handle your workflow parameters, streamlining the method of making, storing, retrieving, and modifying them.

Along with offering an environment friendly parameter administration system, a parameter supervisor additionally offers a spread of options corresponding to parameter creation, storage, retrieval, modification, versioning, entry management, and auditing. These options be certain that your workflow is executed reliably and persistently whereas additionally correctly managing and storing your parameters all through your complete workflow.

The Workflow Orchestrator executes workflows by deciphering the which means of every DSL node and performing the corresponding actions. It manages low-level capabilities corresponding to storing state, interacting with the motion retailer to carry out an motion, listening for callbacks via the occasion retailer, and permitting builders to focus on workflow logic relatively than technical particulars. Journey Platform makes use of Temporal because the underlying workflow engine for state upkeep and orchestration. Temporal helps orchestrate workflows via Temporal Workers.

Builders can incorporate customized performance corresponding to new nodes or edges to broaden platform capabilities, making it easier to create workflows that fulfill the platform’s and customers’ distinctive necessities. Moreover, it helps superior options like parallel execution and computerized retries and enhancing platform reliability and efficiency.

Making certain SLA for processing various kinds of workflows (i.e. transactional and promotional) is essential at scale. Transactional notifications initiated by person motion (e.g. reserving affirmation, visitor/Host messaging, and so on.) have a strict SLA and require increased precedence when in comparison with promotional notifications. To attain this, we now have applied the next at totally different elements of the system:

Occasion pre-processing:

• Pre-filter: As an alternative of passing all of the occasions on to the Workflow Handler, the occasion processor filters out occasions that don’t match the factors. e.g. solely move if the occasion kind is reservation_complete and filter out for all different reservation occasions. This drastically reduces the QPS feeding into the system.
• Mixture excessive QPS occasions: Occasions like searches have a excessive QPS. As an alternative of straight processing, we batch and mixture them over a time window. This reduces the QPS by at the very least a number of orders of magnitude.

Devoted lanes:

• We now have devoted lanes for various classes of workflows via the system. The occasion listener has totally different shopper teams with built-in throttling. The workflow handler has devoted Temporal namespaces for every class and strict limits on the processing QPS, max QPS to the database, and so on.

The Journey Platform empowers non-technical and technical customers to create advanced stateful workflows via a easy drag and drop interface. By leveraging a generic workflow definition DSL, together with motion retailer, occasion retailer, and attribute retailer, the platform facilitates the creation of workflows that reply to real-time occasions, streamlining communication, and enhancing person experiences.

Involved in working at Airbnb? Try these open roles.

Because of Balaji Kalaimani, Davis Wamola, Iris Feng, Jesse Garrison, John Bernardo, Kumar Arjunan, Michael Endelman, Priyank Singhal, Steve Krulewitz, Tej Sudha, Victoria Gryn, Xin Tu, and Zhentao Solar for his or her contributions in constructing Journey Platform.

Because of Sagar Naik and Michael Kinoti for his or her management and supporting us on this Journey.

All product names, logos, and types are property of their respective homeowners. All firm, product and repair names used on this web site are for identification functions solely. Use of those names, logos, and types doesn’t indicate endorsement.