The open-source protocol for creating interactive, data-driven blocks
This document is a working draft
This specification is currently in progress. We’re drafting it in public to gather feedback and improve the final document. If you have any suggestions or improvements you would like to add, or questions you would like to ask, feel free to submit a PR or open a discussion on our GitHub repo.
We have published HASH, an example embedding application with example blocks.
You can see source code for an example block here.
We will be developing more complex blocks which demonstrate the full range of functionality described below.
We welcome feedback and suggestions - please open or contribute to a discussion to do so.
A block type should be packaged and distributed in a form which embedding applications can easily insert into a web page, and accompanied by metadata files describing the structure of data that the block accepts. A block package might be made available via a URL, package manager, or catalog of block types.
A block package MUST contain:
source file or files (e.g. HTML and JavaScript files)
A block package SHOULD contain:
a JSON file describing the properties the block accepts using JSON Schema vocabulary – the ‘block schema’, which can be automatically generated from block code, and which:
block-schema.json
a JSON file containing metadata describing the block, which:
MUST be called block-metadata.json
MUST specify:
name
for the blockexternals
– i.e. libraries which the block depends on but does not include in its package – using the name of the library as it is usually distributed (e.g. via npm), and the expected version (or version range).
For example, a block may rely on React ("externals": [{ "react": "^16.0.0" }]
), but assume that the embedding application will provide it, to save loading it multiple times on a page.index.html
, index.js
) under source
version
of the block, which SHOULD use semantic versioningprotocol
: the applicable block protocol version: currently 0.1
.SHOULD specify:
default
– an object, conforming to the block’s schema, representing
the default data that applications should provide when creating a block, unless (a) the block can handle being provided no data when first instantiated or (b) variants
is provided (see below).icon
– an icon for the block, to be displayed when the user is
selecting from available blocks (as well as elsewhere as appropriate, e.g.
in a website listing the block)author
: a display name for the author of the blockdescription
: a brief description of the blocklicense
: the license the block is made available under (e.g. MIT)MAY specify:
variants
– an array of objects, each with a name
, description
, icon
, and properties
, which represents different variants of the block that the user can create. As a simple example, a ‘header’ block might have variants with the name
‘Heading 1’ and ‘Heading 2’, which start with { level: 1 }
and { level: 2 }
as properties
, respectively.image
– a preview image of the block for users to see it in action before using it. This would ideally have a 3:2 width:height ratio and be a minimum of 900x1170px.Blocks MUST use the data properties specified in their schema, if any.
They can expect these properties to be made available to them by the embedding application, exactly how depending on rendering context.
The data for the block itself will be at the root level of the data made available to it, i.e. a block which had a level
property at the root of the properties in its schema would have a level
property passed to it.
The block’s data represents an ‘entity’ in the system. Alongside the properties declared in its schema, blocks MUST expect certain fields to be provided by the embedding application, and MUST pass these fields back when requesting action with specific entities:
entityId
: an identifier for the entityentityTypeId
: an identifier for the type/class of the entityaccountId
: an identifier for the account/namespace/user to which the entity belongsThese fields MUST be strings. Some MAY be left undefined.
The exact form of these will differ across applications – e.g. some might use human-readable strings, others might use integers (passed as strings) or uuids.
It's vital to have a way of blocks identifying the entities they target for retrieval or modification.
The approach described above attempts to identify a selection of fields which in combination are sufficient to identify an entity for any application, with applications left to decide which to define.
Ideally, blocks would be able to rely on the concatenation of any supplied identifiers as a unique identifier for the entity within that application, so that the block can track unique entities. An alternative is to require in the specification that entityId
is unique in the application, and leave it up to embedding applications to make it so.
We are also considering an alternative approach whereby the entity identifier is a single, opaque object or string which the block has no knowledge of the content of, except that it is sufficient to uniquely identify the entity.
This would have the advantage of the specification not needing to anticipate all the fields an application might need to reliably identify an entity.
It has the disadvantage of potentially losing the explicit recognition of fields which blocks might find useful for their internal workings (e.g. entityTypeId
for grouping entities by type).
We welcome any thoughts or suggestions on this - please open or contribute to a discussion to get involved.
The entity for the block might link to other entities.
For example, a Kanban board block might have a team
property which links to another type of entity, a Team.
Blocks SHOULD expect the following additional fields related to linked data to be passed to them:
a linkedEntities
field containing the entities linked from the block’s entity, and entities linked to from those entities, and so on. This field SHOULD contain all entities which have been resolved in the graph (rather than just those linked directly from the block entity), i.e. these are the nodes in a graph resolved from the block entity to a certain depth (see callout below).
a linkGroups
field which provides the links attached to the block or the entities provided in linkedEntities
, grouped by entity and path, i.e. these are the edges of the graph.
a linkedAggregations
field containing the results of aggregations linked from the block’s entity - these are special links which represent an aggregation operation (e.g. in plain English, "top 10 Cars sorted by age, descending"), which are provided in this field along with the results of the operation at the time of request.
Each entity provided under linkedEntities
or linkedAggregations
will also have identifying fields to be used as arguments when updating those entities.
See linking entities for a discussion of how links are managed.
There must be a limit on what links are followed from a block entity when providing data to it - if any are at all.
The specification currently suggests that at least some links are followed (otherwise there would be no linkedEntities
at all).
It could alternatively specify:
Our own work-in-progress embedding application uses a depth of 1-2 (for individual linked entities and linked aggregations, respectively), which aims to balance speed of block operation (by having more data available immediately) with avoiding resolving tons of data. As we develop more complex blocks we will update our view, and we welcome feedback.
Where available, blocks SHOULD expect and handle an entityTypes
field containing entity type definitions for any entities sent to the blocks, which can be parsed to understand the constraints on user input for each entity (e.g. which fields are editable, what sort of data they accept).
Each entry in the entityTypes
array is a JSON schema object with an additional entityTypeId
field corresponding to the entityTypeId
of the entities it describes the shape of (which MAY be different to the URI value for the standard JSON schema $id
property).
The Block Protocol does not seek to describe or prescribe the shape of particular entities (e.g. what fields a Person
has).
Instead, it seeks to define the block-application interface.
This does, however, mean there is a possibility of competing schemas attempting to describe the same entities, which different blocks using different schema - reducing the portability of blocks.
The ability to translate between schemas would help - e.g. some way expressing an equivalence relationship between properties in different schema. This might be a keyword such as sameAs
or equivalentTo
mapping between schemas and their properties. Then, either blocks or embedding applications could programmatically translate between schemas.
Subject to the permissions granted to them by the embedding application, blocks can expect functions with the names and signatures listed below to be made available to them, i.e. to be passed in along with the properties defined in their schema, or to be otherwise made available in their scope depending on their implementation.
Blocks should have sensible fallbacks for when permissions are denied them, or when functions are absent, for example:
implementing a readonly / display mode for data which they are unable to edit: for example, embedding applications may choose to use blocks to display data as part of a static page, rather than in an editing environment.
hiding or disabling specific controls depending on the specific permissions granted them: for example, hiding a 'delete' button if delete permissions have been denied.
There are various ways in which we may change the approach described here:
There are also many other functions/operations one can imagine being useful for blocks, beyond the basic CRUD operations described below, e.g.
getLocation
Any operations specified in the protocol should be useful in a wide variety of contexts. We welcome ideas!
createEntities<T>(actions: CreateEntitiesAction<T>[]): Promise<T[]>
creates one or more entities
returns: the created entities, i.e. objects inside an array
accepts: a single array of objects (CreateEntitiesAction
), each with the following shape:
accountId?
[string][optional]: the account id of the entity to create.entityTypeId
[string]: the type of entity to create.entityTypeVersionId?
[string][_optional]: specify that this entity is of a particular version of the type (not simply the latest version).data<T>
[object]: the field(s) and value(s) with which to create the entity, i.e. its properties.links?
: [object][optional]: any links to create along with this entity.
See linking entities.selection?
[array of strings][optional]: limit the return to only include these fields on the entity.depth?
[integer][optional]: limit the depth to which linked data in an entity will be resolved.
See linking entities.updateEntities<T>(actions: UpdateEntitiesAction<T>[]): Promise<T[]>
updates one or more entities
returns: the updated entities, i.e. objects inside an array
accepts: a single array of objects (UpdateEntitiesAction
), each with the following shape:
accountId?
[string][optional]: the account id of the entity to update.data<T>
[object]: the fields and values to update on the entity.entityTypeId?
[string][optional]: the type id of the entity to update.entityId
[string]: the id of the entity to update.selection?
[array of strings][optional]: limit the return to only include these fields on the entity.depth?
[integer][optional]: limit the depth to which linked data in an entity will be resolved.
See linking entities.deleteEntities(actions: DeleteEntitiesAction[]): Promise<boolean[]>
deletes one or more entities
returns: an array of boolean
indicating the success of each operation.
accepts: a single array of objects (DeleteEntitiesAction
), each with the following shape:
accountId?
[string][optional]: the account id of the entity to delete.entityTypeId?
[string][optional]: the type id of the entity to delete.entityId
[string]: the id of the entity to delete.getEntities<T>(actions: GetEntitiesAction<T>[]): Promise<T[]>
retrieve one or more entities
returns: the retrieved entities, i.e. objects inside an array.
accepts: a single array of objects (GetEntitiesAction<T>
), each with the following shape:
accountId?
[string][optional]: the account id of the entity to retrieve.entityTypeId?
[string][optional]: the type id of the entity to retrieve.entityId
[string]: the id of the entity to retrieve.selection<T>?
[array of strings][optional]: limit the return to only include these fields on the entity.depth?
[integer][optional]: limit the depth to which linked data in an entity will be resolved.
See linking entities.aggregateEntities(payload?: AggregateEntitiesPayload): Promise<AggregateEntitiesResult>
retrieve a subset of entities of a given type
We are considering moving to the cursor-based Connections pattern for handling pagination, instead of the page-based one described below.
returns: an AggregateEntitiesResult
object
results
: [array]: an array of entitiesoperation
: the aggregation operation applied:entityTypeId?
[string][optional]: the specific type results were limited toentityTypeVersionId?
[string][optional]: the specific version of a type results were limited topageNumber
[integer]: the page number returned.itemsPerPage
[integer]: the number of results per page - i.e. the number of results returned.totalCount
[integer][optional]: the total number of records available for this query.pageCount
[integer][optional]: the total number of pages available for this query.multiFilter
[array][optional]: any filters applied (empty or omitted if none):field
[string]: the field name filtered by.operator
[enum]: the filter operator.
One of IS, IS_NOT, CONTAINS, DOES_NOT_CONTAIN, STARTS_WITH, ENDS_WITH, IS_EMPTY, IS_NOT_EMPTY.value
[string]: the value filtered against.multiSort
[array][optional]: the sort(s) applied (empty or omitted if none):field
[string]: the field sorted on.desc
[boolean]: whether the sort was descending.accepts: an object (AggregateEntitiesPayload
) with the following shape:
accountId?
: [optional]\: the account to retrieve entities fromselection?
[array of strings][optional]: limit the return to only include these fields on the entity.depth?
[integer][optional]: limit the depth to which linked data in an entity will be resolved.
See linking entities.operation?
[object][optional]: a description of the aggregation operation, which contains at least one of the following fields:entityTypeId?
[string][optional]:: limit results to entities of a specific typeentityTypeVersionId?
[string][optional]:: limit results to entities of a specific version of a typepageNumber?
[integer][optional]: the page number to request.itemsPerPage?
[integer][optional]: the number of results to return.multiFilter?
[array][optional]: filter entities by a given field value:field
[string]: the field name to filter by.operator
[enum]: the filter operator.
One of IS, IS_NOT, CONTAINS, DOES_NOT_CONTAIN, STARTS_WITH, ENDS_WITH, IS_EMPTY, IS_NOT_EMPTY.value
[string]: the value to match against.multiSort?
[array][optional]: specify how to sort results by providing one or more objects with the following shape:field
[string]: the field name to sort on.desc?
[boolean][optional]: whether to sort descending.The functions defined above return entity data, but block authors should note that in many implementations the embedding application will re-render a block with new entity data whenever it is updated (whether by the block or some other actor), e.g. the block will automatically get sent new data via props when any entity it has previously received via props is updated.
Where supported and permitted by the embedding application, blocks SHOULD be provided with the following functions to work with entity types, i.e. data models.
createEntityTypes(actions: CreateEntityTypesAction[]): Promise<EntityType[]>
creates one or more entity types.
returns: the created entity types, i.e. objects inside an array.
An EntityType
is a JSON schema object with an additional entityTypeId
field and optional accountId
field.
accepts: a single array of objects (CreateEntityTypesAction<T>
), each with the following shape:
accountId?
[string][optional]: the account to create the entity type in.schema
[object]: the JSON schema
for the entity type.updateEntityTypes(actions: UpdateEntityTypesAction[]): Promise<EntityType[]>
updates one or more entity types.
returns: the updated entity types
accepts: a single array of objects (UpdateEntityTypesAction<T>
), each with the following shape:
accountId?
[string][optional]: the account of the entity type to update.entityTypeId
[string]: the id of the entity type to update.schema
[object]: the JSON schema for the entity type.deleteEntityTypes(actions: DeleteEntityTypesAction[]): Promise<boolean[]>
deletes one or more entity types.
returns: an array of boolean indicating the success of each operation.
accepts: a single array of objects (DeleteEntityTypesAction<T>
), each with the following shape:
accountId?
[string][optional]: the account of the entity type to delete.entityTypeId
[string]: the entity type to delete.getEntityTypes(actions: GetEntityTypesAction[]): Promise<EntityType[]>
retrieves one or more entity types.
returns: the retrieved entity types, i.e. objects inside an array.
accepts: a single array of objects (GetEntityTypesAction<T>
), each with the following shape:
accountId?
[string][optional]: the account of the entity type to retrieve.entityTypeId
[string]: the entity type to retrieve.aggregateEntityTypes(payload: AggregateEntityTypesPayload): Promise<AggregateEntitiesResult>
retrieve one or more entity types.
returns: an AggregateEntitiesResult
, where the results
field contains an array of entity types.
accepts: an object (AggregateEntityTypesPayload
), with the following shape:
accountId?
[string][optional]: the account of the entity types to retrieve.Another special set of functions provided to blocks relate to managing links between entities.
When creating or updating an entity’s data, including its own, blocks will often wish to express that a certain property on an entity should be a reference to another entity.
A block may also wish to link one of its properties to a particular aggregation of entities.
In order to create a reference to a separate entity or entities as the desired value of a particular field, blocks SHOULD create a Link
, which:
MUST contain:
sourceEntityId
[string]: the entityId
of the source entity.path
[string]: the path to the field on the source entity this link is conceptually made on, expressed as a JSON path.destinationEntityId
[string] – the id of a single entity the link is made to, ORoperation
– an aggregation operation which the embedding application should resolve the link to, following the structure of the operation
object described above.MAY contain
destinationEntityAccountId?
: [string][optional]: the accountId
of the destination entity or account to aggregate entities from.sourceEntityVersionId?
[string][optional]: optionally specify that this link is only from a specific version.sourceAccountId?
: [string][optional]:: the accountId
of the source entity.sourceEntityTypeId?
: [string][optional]:: the entityTypeId
of the source entity.if destinationEntityId
is defined, MAY contain:
destinationEntityVersionId:
[string][optional]: to pin the link to a specific version of the destination entity.destinationEntityTypeId?
: [string][optional]: the entityTypeId
of the destination entity.index
[integer]: the position of this link in a list (for where ordering of links is important).Once created, a Link
includes a linkId
.
Example 1. creating a Link
with the following data indicates that this particular user should be linked to a company with id company1
, and that the link conceptually is made on the user’s employer
field:
{
"sourceEntityId": "user1",
"destinationEntityId": "company1",
"path": "employer"
}
When delivering data to blocks the resolved data for entity company1
will be provided separately to user1
, in the linkedEntities
array, rather than injected into the properties of the user, and the link itself available in the linkGroups
array provided to the block.
Example 2. creating a Link
with the following data indicates that this particular table should be linked to the top 10 sales by value, and that the link is conceptually made on the table’s rows
field:
{
"sourceEntityId": "table1",
"path": "rows",
"aggregate": {
"entityTypeId": "sales",
"multiSort": [{ "field": "value", "desc": true }],
"itemsPerPage": 10,
"pageNumber": 1
}
}
When delivering the data this data would be provided alongside the table, in the linkedAggregations
array.
Links between specific entities - edges in the graph - will be provided to a block under a linkGroups
field, which is an array of objects,
each of which specifies a source entity, a path (field name), and the links on that path.
{
"sourceEntityId": "user1",
"path": "company",
"links": [
{
"sourceEntityId": "user1",
"destinationEntityId": "company1",
"path": "company"
}
]
}
N.B. this data structure has been chosen to allow for later pagination of links on a field.
The entities linked to - the nodes in the graph - will be provided under linkedEntities
(and the entities they link onwards to, depending on the depth the graph is resolved to from the starting entity).
An entry in linkedAggregations follows the shape of the AggregateEntitiesResult
object,
with the addition of:
sourceAccountId?
[string][optional]: the accountId
of the source entity.sourceEntityId
[string]: the entityId
of the source entity.sourceEntityTypeId?
[string][optional]: the entityTypeId
of the source entity.path
: [string]: the path
on the source entity to which this aggregation is linked.To create, update and delete links between entities, blocks SHOULD expect the following functions to be made available to them:
createLinks(actions: CreateLinksAction[]): Promise<Link[]>
creates one or more links.
returns: the created links, i.e. objects inside an array (now with linkId
)
accepts: a single array of objects (CreateLinksAction
) -
each object is a Link
, omitting linkId
.
updateLinks(actions: UpdateLinksAction[]): Promise<Link[]>
updates one or more links.
returns: the updated links, i.e. objects inside an array
accepts: a single array of objects (UpdateLinksAction
), each with the following shape:
linkId
[string]: the id of the link to update.data
[object]: the Link
to overwrite the existing one with.deleteLinks(actions: DeleteLinksAction[]): Promise<boolean[]>
deletes one or more links.
returns: an array of boolean indicating the success of each operation.
accepts: a single array of objects (DeleteLinksAction
), each with the following shape:
sourceAccountId?
[string][optional]: the accountId
of the source entity.sourceEntityId
[string][optional]: the entityId
of the source entity.linkId
[string]: the entity type to delete.getLinks(actions: GetLinksAction[]): Promise<Link[]>
retrieve one or more links.
returns: the retrieved links, i.e. objects inside an array.
accepts: a single array of objects (GetLinksAction
), each with the following shape:
linkId
[string]: the link to retrieve.Where blocks wish to express in their schema – or in the schema of any other entity type – that the value of a field should be a link to another entity, they can use the JSON schema $ref keyword when describing the accepted types for the field. The value of the $ref should be the value of $id in the JSON schema for the target type.
Any linked entities MUST be provided by embedding applications either:
linkedEntities
field, where the expected type is a single entity or a list of specific entities (with the linkGroups
field describing the links)linkedAggregations
field, where the expected type is the result of an aggregation operation (e.g. Top 10 X, ordered by Y).
See linking entities for more on these fields.Where blocks wish to express that a property in a schema is the inverse of another property, they can use an inverseOf
keyword with a $ref pointing to the relevant schema and property. Embedding applications can use inverseOf
declarations to resolve inverse links without blocks needing to create them in both directions.
E.g. to express that a company’s employees
field is the inverse of users’ employer
field:
{
"$id": "https://example.com/schemas/company",
"type": "object",
"properties": {
"employees": {
"type": "array",
"items": {
"type": { "$ref": "https://example.com/schemas/user" }
},
"inverseOf": {
"$ref": "https://example.com/schemas/user#/properties/employer"
}
}
}
}
Limiting linked data returnedWhen requesting entity data via a block protocol function, blocks MAY include a depth
field which will specify how many levels of linked entity data to resolve, to avoid expensive queries that pull in unneeded data from an extensive entity graph.
For example, a depth of 2 on a Person would resolve their linked Employer, and their Employer’s linked Location, but no further.
A depth of 0 would resolve no links to other entities.
A block can expect the following fields to be made available to it, whether passed in as props or via another method appropriate to their rendering strategy:
// data representing the block entity itself
entityId
entityTypeId
accountId
// ...plus any keys declared in the block’s schema
// data representing entities linked from the block and onwards
linkedEntities
linkedAggregations
linkGroups
// functions
createEntities
updateEntities
deleteEntities
getEntities
aggregateEntities
createEntityTypes
updateEntityTypes
deleteEntityTypes
getEntityTypes
aggregateEntityTypes
createLinks
updateLinks
deleteLinks
getLinks
// other
styleVariables
Embedding applications will wish to restrict the data that blocks can restrict or modify.
This will require several things we are developing:
Where blocks interact with third-party data stores, i.e. they send data for storage outside the embedding application, they SHOULD where possible keep the entity data in the embedding application in sync, for example by:
creating entities via the functions described above at the same time as creating records in other data stores
reflecting any changes to entities provided by the embedding application if the user takes action to edit them in the block UI, by updating entities via the functions described above
Where changes to relevant entity data can occur in both the embedding application and the third-party data store even when the block is not being used, additional synchronization outside the block will be required to ensure consistent user data.
We are considering options for blocks reporting on user actions within them, both to allow the embedding application to track activity, and to be able to indicate user focus to other users where the application implements collaborative/multiplayer editing.
Potential options include:
passing a reportUserAction
function to blocks, to report on keypresses, drags, etc.
passing a usersFocus
property to blocks containing an array of focus objects, each indicating where different users are focused, to allow the block to render indicators.
handling tracking user focus and rendering focus indicators outside of blocks.
Relatedly, we could require blocks to send finer-grained change steps when updating entities, to more precisely understand what actions users are taking within blocks and better reconcile them with other users' actions.
While embedding applications can handle displaying an interface for reloading blocks at particular earlier versions, we will specify a way of communicating to blocks that (a) an earlier version is being displayed, and (b) the difference with the current version would allow blocks to implement visual diffs and so on.
We want to facilitate users leaving comments on elements within blocks.
This could be
managed entirely outside the block, e.g. by a wrapper around the block which provides a context menu to users for adding comments on blocks – which avoids blocks having to have any knowledge of commenting, but could interfere with how the block wants to respond to user input,or
managed by providing functions to blocks to trigger a comment attached to specific elements in blocks – which allows blocks to have control over how and to what element the user is able to attach comments, but means that blocks have to implement ‘offer comment option’ behavior.
Blocks SHOULD provide at least basic visual styling to allow them to be embedded and used without modification by any web application.
Blocks SHOULD use the theme variables provided under the styleVariables
object and listed in Appendix A as property values where appropriate, and provide fallback values in case the embedding application does not define them.
Previous
Anyone with an existing application who wants to embed semantically-rich, reusable blocks in their product can use the protocol. Improve your app’s utility and tap into a world of structured data with no extra effort, for free.
Any developer can build and publish blocks to the global registry for other developers to use. Create blocks that solve real-world problems, and contribute to an open source community changing the landscape of interoperable data.