How overlays work
An overlay is a service that tracks the transactions of one app or protocol. Your app sends it the transactions it creates, the overlay keeps the outputs that follow your rules, and your app queries it later to read them back. It is how a BSV blockchain app keeps track of its own data, such as tokens, records or file references, without indexing every block on the chain. This page was written against @bsv/sdk 3.2.1.
Why apps track their own transactions
A node keeps the whole chain, but it does not know which transactions belong to your app. An overlay starts from the other end: your app already has its transactions, because its wallet created them.
- Index the chain yourself
- Processes every transaction on the network
- You run the indexer, or trust someone else’s
- Trusts that the index left nothing out
- Costs grow with the whole network
- Use an overlay
- Processes only the transactions your app sends
- The overlay checks each one, usually with SPV
- Applies your own rules
- Costs grow with your app
The state of a BSV blockchain app lives in UTXOsUnspent transaction output, not in an account that a node updates. An output can be spent only once, and the miners reject a second spend. So when the overlay sees that an output it tracks was spent, it knows the state changed, and an app that holds an old copy finds out as soon as it tries to spend that output.
The two parts of an overlay
An overlay serviceBuilt from topic managers, which decide what is admitted, and lookup services, which query what was admitted. pairs a topic manager with a lookup service, and one overlay host can run many pairs. Their names start with tm_ and ls_, as set by BRC-87BSV Request for Comments 87, the naming rules for topic managers and lookup services:
Overlay host
tm_Applies the rules of your protocol and admits only the outputs that pass, for example a token with a valid format.
ls_Indexes the admitted outputs and answers queries about them, for example all the tokens of one asset.
How a transaction goes through an overlay
A transaction goes through an overlay in five steps. The submission side is defined in BRC-22BSV Request for Comments 22, overlay network data synchronization and the query side in BRC-24BSV Request for Comments 24, overlay network lookup services.
Your app sends the transaction
Your app builds the transaction with its wallet and sends it to the overlay host in BEEFBackground Evaluation Extended Format (BRC-62), with the names of the topics it is meant for.
The topic managers admit outputs
The host verifies the transaction, and each topic manager picks the outputs it admits.
The host answers with a STEAK
The STEAKSubmitted Transaction Execution AcKnowledgment lists the outputs each topic admitted.
The lookup service indexes them
The lookup service indexes the admitted outputs. When a later transaction spends one of them, the topic manager either removes it from the topic or keeps it as history.
Your app queries the lookup service
Your app queries the lookup service by name and gets back the matching outputs as BEEF, so it can verify them before it uses the data.
How your app finds an overlay host
An overlay is shared infrastructure, and only a host that runs your topic can track it. Your app does not need the address of that host in advance: hosts announce what they run with two kinds of advertisement, defined in BRC-88BSV Request for Comments 88, overlay services synchronization architecture:
- A SHIPServices Host Interconnect Protocol advertisement says that a host runs a topic, so a transaction can be sent to every host that tracks it. In
@bsv/sdk,TopicBroadcasteruses it to send a transaction to the hosts of its topics - A SLAPServices Lookup Availability Protocol advertisement says that a host runs a lookup service, so a query can find a host that answers it. In
@bsv/sdk,LookupResolveruses it to send a query to a host of the service
Both classes find the hosts through SLAP trackers, the hosts that keep the list of advertisements for a network. Each network has its own trackers, so your app looks on the network where its host runs.
When no public host runs your topic, you run one. @bsv/overlay-express runs a host as a Node server, and @bsv/overlay-topics has ready topic managers and lookup services, for tokens, identity certificates, key-value storage and file references, among others.
Related articles
To learn more about overlays on the BSV blockchain, review these articles: