Run an overlay with @bsv/overlay-express
Use @bsv/overlay-express to run an overlay serviceA service that indexes an app's transactions. Topic managers decide which outputs it admits, and lookup services answer queries about them for your own app on an Express server. The package wires the overlay engine to the storage, the network settings and the HTTP routes. You register your topics and lookup services, then start the server. For the full walkthrough, with a transaction sent to the host and looked up, follow Track transactions with overlays.
Install
The latest version on npm is 2.8.0, with the status Stable. It needs Node.js 22 or newer and @bsv/sdk. This page was written against @bsv/overlay-express 2.8.0, @bsv/overlay-topics 2.0.3 and @bsv/sdk 4.0.1. For a local host, add the topics package, which is Experimental, and the storage packages. In a project folder with "type": "module" in its package.json, run:
npm install @bsv/overlay-express@2.8.0 @bsv/overlay-topics@2.0.3 @bsv/sdk@4.0.1 mongodb-memory-server better-sqlite3 knexWhat the package includes
The package connects the overlay engine to storage, HTTP routes and the network:
| Piece | What it does |
|---|---|
OverlayExpress | Creates the server and exposes the configure methods and start |
| Topic managers and lookup services | Decide which transaction outputs the host admits and answer queries about them. Prebuilt ones, such as tm_helloworld, come from @bsv/overlay-topics |
| Storage | A SQL database through Knex for the host’s own data, and MongoDB for the lookup services |
| HTTP routes | /submit and /lookup for apps, health routes, a web UI, and /admin routes for the operator |
| Network services | GASPGraph Aware Sync Protocol, how overlay hosts copy each other's data synchronization with other hosts, and ARC or BSV BroadcasterServices that broadcast transactions and return proofs that they were mined for broadcasting and proofs |
Example on testnet
The example runs a local host on testnet with the tm_helloworld topic and the ls_helloworld lookup service. It keeps everything in memory, so it starts empty each time.
Write the host
Create overlay-host.mjs in your project folder.
The script generates a new host key each time it starts, which suits this local example. For a host you keep running, load the key from an environment variable and never paste it into your code.
The host script is:
import OverlayExpress from '@bsv/overlay-express'
import { HelloWorldTopicManager, createHelloWorldLookupService } from '@bsv/overlay-topics'
import { MongoMemoryServer } from 'mongodb-memory-server'
import { PrivateKey } from '@bsv/sdk'
const mongo = await MongoMemoryServer.create()
const server = new OverlayExpress(
'my-overlay',
PrivateKey.fromRandom().toHex(),
'localhost:8080',
process.env.ADMIN_TOKEN
)
server.configurePort(8080)
server.configureNetwork('test')
await server.configureKnex({
client: 'better-sqlite3',
connection: { filename: ':memory:' },
useNullAsDefault: true
})
await server.configureMongo(mongo.getUri())
server.configureTopicManager('tm_helloworld', new HelloWorldTopicManager())
await server.configureLookupServiceWithMongo('ls_helloworld', db => createHelloWorldLookupService(db))
await server.configureEngine()
await server.start()What the key lines do:
OverlayExpresstakes a name, the key that signs the host’s advertisements, the address it announces withouthttps://, and the optional admin token.configureNetwork('test')runs the host on testnet. Without it the host runs on Mainnet.better-sqlite3keeps the SQL database in memory, andmongodb-memory-serverstarts a temporary MongoDB for development.
Start the host
Generate an admin token of at least 32 bytes and start the host with it. The command works in Git Bash or any Unix shell:
export ADMIN_TOKEN=$(node -p "require('crypto').randomBytes(24).toString('hex')")
node overlay-host.mjsThe host prints a long startup log. It is ready when the last line is:
my-overlay is ready and listening on local port 8080The log also has a line that starts with Advertiser not initialized for FQDN localhost:8080. This is expected on localhost, where the host does not announce itself to other hosts. Next, the log shows Failed to GASP sync and No competent mainnet hosts found by the SLAP trackers, because a local host has no peers to synchronize with. Both messages are harmless, and the host makes no outside connection while idle on testnet.
Check the host
Open http://localhost:8080/health. The answer starts with {"status":"ok","live":true,"ready":true, and names the network, test.
Configuration options
The sections below cover the settings of the host, the order of the calls and the admin routes.
Settings and defaults
These settings control the network, the port, the admin access and the synchronization of the host:
| Option | Default | Effect |
|---|---|---|
configureNetwork | main | Chooses main, test or ttn. With the default, the host contacts public Mainnet hosts at startup and synchronizes from them |
configurePort | 3000 | The port the server listens on |
| Admin token | A random UUID | The fourth argument of the constructor, between 32 and 16,384 bytes. Read the generated one with server.getAdminToken() |
configureEnableGASPSync | true | Copies data from other overlay hosts when the host starts |
configureArcApiKey, configureArcade | Not set | Broadcasting and proofs through ARC or BSV Broadcaster, which configureArcade configures. Without a provider the host disables the ingest route for their callbacks |
Order of the calls and admin routes
Call the methods in this order:
configureKnexandconfigureMongoconfigureTopicManagerandconfigureLookupServiceWithMongoconfigureEnginestart
Each step fails with its own message when an earlier one is missing, for example You must configure your SQL database with the .configureKnex() method first!. The host needs both databases, even when none of your own services uses MongoDB.
Every /admin route needs the admin token, except /admin/config, which answers without one and returns the adminIdentityKey and the nodeName of the host. Send the token as a bearer token, Authorization: Bearer YOUR_ADMIN_TOKEN, and replace YOUR_ADMIN_TOKEN with your token. The host answers like this:
| Request | Answer |
|---|---|
/health, /listTopicManagers or /admin/config, with no token | 200 |
Any other /admin route, with no token | 401 |
Any other /admin route, with a wrong token | 403 |
Any other /admin route, with the right token | 200 |
Limitations
Check these two limits before you run an app on testnet:
- No public testnet overlay host or tracker works, so a testnet app runs its own host, as in this example.
- Other users cannot reach a host on
localhost. To announce itself to other hosts through SHIPServices Host Interconnect Protocol and SLAPServices Lookup Availability Protocol, a host needs a public address instead oflocalhost.
Related articles
To learn more about overlays on the BSV blockchain, review these articles: