# Replaying Historical Data HTTP and WebSocket endpoints for tick-by-tick historical market data replay in both exchange-native and normalized formats. ## Exchange-native market data APIs Exchange-native market data API endpoints provide historical data in [exchange-native format](https://docs.tardis.dev/faq/data#what-is-a-difference-between-exchange-native-and-normalized-data-format). The main difference between HTTP and WebSocket endpoints is the logic of requesting data: * [HTTP API](#http-get-replay-options-options) accepts [request options](#replay-options) payload via query string param * [WebSocket API](#websocket-ws-replay-exchange-exchange-and-from-fromdate-and-to-todate) accepts exchanges' specific 'subscribe' messages that define what data will be then "replayed" and send to WebSocket client ### `HTTP GET` /replay?options={options} Returns historical market data messages in [exchange-native format](https://docs.tardis.dev/faq/data#what-is-a-difference-between-exchange-native-and-normalized-data-format) for given replay options query string param. Single streaming HTTP response returns data for the whole requested time period as [NDJSON](http://ndjson.org/). {% hint style="info" %} In our preliminary benchmarks on AMD Ryzen 7 3700X, 64GB RAM, [HTTP /replay](#http-get-replay-options-options) API endpoint was returning \~700 000 messages/s (already locally cached data). {% endhint %} {% hint style="warning" %} Each replay request can trigger many parallel HTTP requests to the upstream Tardis API (one per minute of data per channel/symbol combination). Running multiple concurrent replay processes can quickly exceed [API rate limits](https://docs.tardis.dev/faq/general#are-there-any-rate-limits-for-the-api), causing stalls without visible error messages. {% endhint %} {% tabs %} {% tab title="Python" %} ```python import asyncio import aiohttp import json import urllib.parse async def replay_via_tardis_machine(replay_options): timeout = aiohttp.ClientTimeout(total=0) async with aiohttp.ClientSession(timeout=timeout) as session: # url encode as json object options encoded_options = urllib.parse.quote_plus(json.dumps(replay_options)) # assumes tardis-machine HTTP API running on localhost:8000 url = f"http://localhost:8000/replay?options={encoded_options}" async with session.get(url) as response: # otherwise we may get line to long errors response.content._high_water = 100_000_000 # returned data is in NDJSON format http://ndjson.org/ # each line is separate message JSON encoded async for line in response.content: yield line async def main(): lines = replay_via_tardis_machine( { "exchange": "binance", "from": "2024-03-01", "to": "2024-03-02", "filters": [ {"channel": "trade", "symbols": ["btcusdt"]}, {"channel": "depth", "symbols": ["btcusdt"]}, ], } ) async for line in lines: message = json.loads(line) # localTimestamp string marks timestamp when message was received # message is a message dict as provided by exchange real-time stream print(message["localTimestamp"], message["message"]) asyncio.run(main()) ``` {% hint style="info" %} See also official Tardis.dev [Python client](https://docs.tardis.dev/python-client/quickstart) library. {% endhint %} {% endtab %} {% tab title="Node.js" %} ```javascript import split2 from 'split2' import { Readable } from 'stream' const serialize = options => { return encodeURIComponent(JSON.stringify(options)) } async function* replayViaTardisMachine(options) { // assumes tardis-machine HTTP API running on localhost:8000 const url = `http://localhost:8000/replay?options=${serialize(options)}` const response = await fetch(url) // returned data is in NDJSON format http://ndjson.org/ // each line is separate message JSON encoded // split response body stream by new lines const lines = Readable.fromWeb(response.body).pipe(split2()) for await (const line of lines) { yield line } } const options = { exchange: 'binance', from: '2024-03-01', to: '2024-03-02', filters: [ { channel: 'trade', symbols: ['btcusdt'] }, { channel: 'depth', symbols: ['btcusdt'] } ] } const lines = replayViaTardisMachine(options) for await (const line of lines) { // localTimestamp string marks timestamp when message was received // message is a message object as provided by exchange real-time stream const { message, localTimestamp } = JSON.parse(line) console.log(message, localTimestamp) } ``` {% hint style="info" %} See also official Tardis.dev [Node.js client](https://docs.tardis.dev/node-client/quickstart) library. {% endhint %} {% endtab %} {% tab title="cURL" %} ```bash curl -g 'localhost:8000/replay?options={"exchange":"binance","filters":[{"channel":"trade","symbols":["btcusdt"]},{"channel":"depth","symbols":["btcusdt"]}],"from":"2024-03-01","to":"2024-03-02"}' ``` {% embed url="" %} Click to see API response in the browser as long as tardis-machine is running on localhost:8000 {% endembed %} {% endtab %} {% tab title="Your preferred language" %} We're working on providing more samples and dedicated client libraries in different languages, but in the meanwhile to consume [HTTP /replay](#http-get-replay-options-options) API responses in your language of choice, you should: 1. Provide url encoded JSON [options object](#replay-options) via options query string param when sending HTTP request 2. Parse HTTP response stream line by line as it's returned - buffering in memory whole response may result in slow performance and memory overflows 3. Parse each response line as JSON containing messages in [exchange-native format](#response-format) {% endtab %} {% endtabs %} #### Replay options [HTTP /replay](#http-get-replay-options-options) endpoint accepts required options query string param in **url encoded JSON format**. | name | type | default | description | | ------------------------------- | ---------------------------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **exchange** | string | - | requested exchange id - use [/exchanges HTTP API](https://docs.tardis.dev/api/http-api-reference#exchanges) to get list of valid exchanges ids | | **filters** | {channel:string, symbols?: string\[]}\[] | \[] | optional filters of requested historical data feed - check [historical data details](https://docs.tardis.dev/historical-data-details/overview) for each exchange and [/exchanges/:exchange HTTP API](https://docs.tardis.dev/api/http-api-reference#exchanges-exchange) to get allowed channels and symbols for requested exchange | | **from** | string | - | replay period start date (UTC) in a [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format, e.g., `2019-04-01` | | **to** | string | - | replay period end date (UTC) in a [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format, e.g., `2019-04-02` | | **withDisconnects** | boolean (optional) | undefined | when set to `true`, response includes empty lines (`\n`) that mark events when real-time WebSocket connection that was used to collect the historical data got disconnected | | **waitWhenDataNotYetAvailable** | boolean or number (optional) | undefined | when set to `true`, waits for data that is not yet available — useful when replaying near real-time data. Defaults to a 30-minute offset. When set to a number, specifies the offset in minutes (minimum effective value is 6 minutes). | | **autoCleanup** | boolean (optional) | undefined | when set to `true`, automatically removes cached data from disk after it has been processed. Not safe for concurrent replay — see [warning](#replay-options) | | **withMicroseconds** | boolean (optional) | undefined | when set to `true`, `localTimestamp` in response is returned with microsecond precision | When `symbols` array is empty or omitted in filters, data for all active symbols is returned. {% hint style="info" %} **Disconnect markers**: An empty line in raw replay (or a `disconnect` message in normalized replay) indicates that the WebSocket connection used during data collection was interrupted. After a reconnect, exchanges typically re-send initial snapshots, so duplicate snapshot messages are expected following a disconnect marker. Disconnect markers apply to the entire connection, not individual channels. {% endhint %} {% hint style="warning" %} **`autoCleanup` concurrency**: avoid using `autoCleanup` with concurrent replay — the cache path is keyed by exchange, a hash of the filters, and the calendar day, so any concurrent jobs sharing those three components will conflict (even with non-overlapping time windows within the same day) and cause file-not-found errors. Clean the cache manually after all jobs complete instead. {% endhint %} #### **Response format** Streamed HTTP response provides data in NDJSON format (new line delimited JSON) - each response line is a JSON with market data message in [exchange-native format](https://docs.tardis.dev/faq/data#what-is-a-difference-between-exchange-native-and-normalized-data-format) plus local timestamp: * `localTimestamp` - date when message has been received in ISO 8601 format * `message` - JSON with exactly the same format as provided by requested exchange real-time feeds **Sample response** ```javascript {"localTimestamp":"2024-03-01T00:00:00.001409Z","message":{"stream":"btcusdt@trade","data":{"e":"trade","E":1709251199998,"s":"BTCUSDT","t":3445374963,"p":"61130.98000000","q":"0.00145000","b":25230662315,"a":25230662808,"T":1709251199998,"m":true,"M":true}}} {"localTimestamp":"2024-03-01T00:00:00.001417Z","message":{"stream":"btcusdt@trade","data":{"e":"trade","E":1709251199999,"s":"BTCUSDT","t":3445374964,"p":"61130.98000000","q":"0.00182000","b":25230662315,"a":25230662809,"T":1709251199998,"m":true,"M":true}}} {"localTimestamp":"2024-03-01T00:00:00.001423Z","message":{"stream":"btcusdt@trade","data":{"e":"trade","E":1709251199999,"s":"BTCUSDT","t":3445374965,"p":"61130.98000000","q":"0.00034000","b":25230662315,"a":25230662810,"T":1709251199998,"m":true,"M":true}}} {"localTimestamp":"2024-03-01T00:00:00.001429Z","message":{"stream":"btcusdt@trade","data":{"e":"trade","E":1709251200000,"s":"BTCUSDT","t":3445374966,"p":"61130.98000000","q":"0.00182000","b":25230662315,"a":25230662811,"T":1709251199999,"m":true,"M":true}}} {"localTimestamp":"2024-03-01T00:00:00.006791Z","message":{"stream":"btcusdt@trade","data":{"e":"trade","E":1709251200000,"s":"BTCUSDT","t":3445374967,"p":"61130.98000000","q":"0.00038000","b":25230662315,"a":25230662812,"T":1709251199999,"m":true,"M":true}}} ``` ### `WebSocket` /ws-replay?exchange={exchange}\&from={fromDate}\&to={toDate} Exchanges' WebSocket APIs are designed to publish real-time market data feeds, not historical ones. Tardis-machine [WebSocket /ws-replay](#websocket-ws-replay-exchange-exchange-and-from-fromdate-and-to-todate) API fills that gap and allows "replaying" historical market data from any given past point in time with the same data format and 'subscribe' logic as real-time exchanges' APIs. In many cases existing exchanges' WebSocket clients can be used to connect to this endpoint just by changing URL, and receive historical market data in [exchange-native](https://docs.tardis.dev/faq/data#what-is-a-difference-between-exchange-native-and-normalized-data-format) format for date ranges specified in URL query string params. After connection is established, client has 2 seconds to send subscriptions payloads and then market data replay starts. If two clients connect at the same time requesting data for different exchanges and provide the same [session key via query string param](#query-string-params), then data being send to those clients will be synchronized (by local timestamp). {% hint style="info" %} In our preliminary benchmarks on AMD Ryzen 7 3700X, 64GB RAM, [WebSocket /ws-replay](#websocket-ws-replay-exchange-exchange-and-from-fromdate-and-to-todate) API endpoint was sending \~500 000 messages/s (already locally cached data). {% endhint %} {% tabs %} {% tab title="Python" %} ```python import asyncio import aiohttp import json async def main(): WS_REPLAY_URL = "ws://localhost:8001/ws-replay" URL = f"{WS_REPLAY_URL}?exchange=binance&from=2024-03-01&to=2024-03-02" async with aiohttp.ClientSession() as session: async with session.ws_connect(URL) as websocket: await websocket.send_str( json.dumps( { "method": "SUBSCRIBE", "params": ["btcusdt@trade", "btcusdt@depth@100ms"], "id": 1, } ) ) async for msg in websocket: print(msg.data) asyncio.run(main()) ``` {% hint style="info" %} You can also try using existing WebSocket client by changing URL endpoint to the one shown in the example above. {% endhint %} {% endtab %} {% tab title="Node.js" %} ```javascript import WebSocket from 'ws' const WS_REPLAY_URL = 'ws://localhost:8001/ws-replay' const ws = new WebSocket( `${WS_REPLAY_URL}?exchange=binance&from=2024-03-01&to=2024-03-02` ) ws.onmessage = message => { console.log(message.data) } ws.onopen = () => { ws.send( JSON.stringify({ method: 'SUBSCRIBE', params: ['btcusdt@trade', 'btcusdt@depth@100ms'], id: 1 }) ) } ``` {% hint style="info" %} You can also use existing WebSocket client by changing URL endpoint to the one shown in the example above. {% endhint %} {% endtab %} {% tab title="Your preferred language" %} As long as you already use existing WebSocket client that connects to and consumes real-time exchange market data feed, in most cases you can use it to connect to [/ws-replay](#websocket-ws-replay-exchange-exchange-and-from-fromdate-and-to-todate) API as well just by changing URL endpoint. {% endtab %} {% endtabs %} #### **Query string params** | name | type | default | description | | ------------ | ----------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **exchange** | string | - | requested exchange id - use [/exchanges HTTP API](https://docs.tardis.dev/api/http-api-reference#exchanges) to get list of valid exchanges ids | | **from** | string | - | replay period start date (UTC) in a [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format, e.g., `2019-04-01` | | **to** | string | - | replay period end date (UTC) in a [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format, e.g., `2019-04-02` | | **session** | string (optional) | undefined | optional replay session key. When specified and multiple clients use it when connecting at the same time then data being send to those clients is synchronized (by local timestamp). | ## Normalized market data APIs Normalized market data API endpoints provide data in [unified format](https://docs.tardis.dev/tardis-machine/data-types) across all supported exchanges. Both [HTTP /replay-normalized](#http-get-replay-normalized-options-options) and [WebSocket /ws-replay-normalized](#websocket-ws-replay-normalized-options-options) APIs accept the same replay options payload via query string param. It's mostly a matter of preference when choosing which protocol to use, but [WebSocket /ws-replay-normalized API](#websocket-ws-replay-normalized-options-options) also has its real-time counterpart [/ws-stream-normalized](https://docs.tardis.dev/streaming-real-time-data#websocket-ws-stream-normalized-options-options), which connects directly to exchanges' real-time WebSocket APIs. This opens the **possibility of seamless switching between real-time streaming and historical normalized market data replay**. ### `HTTP GET` /replay-normalized?options={options} Returns historical market data for [data types](#replay-normalized-options) specified via query string. Single streaming HTTP response returns data for the whole requested time period as [NDJSON](http://ndjson.org/). See [supported data types](https://docs.tardis.dev/tardis-machine/data-types) which include normalized [trade](https://docs.tardis.dev/data-types#trade), [order book change](https://docs.tardis.dev/data-types#book_change), [customizable order book snapshots](https://docs.tardis.dev/data-types#book_snapshot_-number_of_levels-_-snapshot_interval-time_unit) etc. {% hint style="info" %} In our preliminary benchmarks on AMD Ryzen 7 3700X, 64GB RAM, [HTTP /replay-normalized](#http-get-replay-normalized-options-options) API endpoint was returning \~100 000 messages/s and \~50 000 messages/s when order book snapshots were also requested. {% endhint %} {% tabs %} {% tab title="Python" %} ```python import asyncio import aiohttp import json import urllib.parse async def replay_normalized_via_tardis_machine(replay_options): timeout = aiohttp.ClientTimeout(total=0) async with aiohttp.ClientSession(timeout=timeout) as session: # url encode as json object options encoded_options = urllib.parse.quote_plus(json.dumps(replay_options)) # assumes tardis-machine HTTP API running on localhost:8000 url = f"http://localhost:8000/replay-normalized?options={encoded_options}" async with session.get(url) as response: # otherwise we may get line to long errors response.content._high_water = 100_000_000 # returned data is in NDJSON format http://ndjson.org/ streamed # each line is separate message JSON encoded async for line in response.content: yield line async def main(): lines = replay_normalized_via_tardis_machine( { "exchange": "binance", "from": "2024-03-01", "to": "2024-03-02", "symbols": ["btcusdt"], "withDisconnectMessages": True, # other available data types examples: # 'book_snapshot_10_100ms', 'derivative_ticker', 'quote', # 'trade_bar_10ms', 'trade_bar_10s' "dataTypes": ["trade", "book_change", "book_snapshot_10_100ms"], } ) async for line in lines: normalized_message = json.loads(line) print(normalized_message) asyncio.run(main()) ``` {% endtab %} {% tab title="Node.js" %} ```javascript import split2 from 'split2' import { Readable } from 'stream' const serialize = options => { return encodeURIComponent(JSON.stringify(options)) } async function* replayNormalizedViaTardisMachine(options) { // assumes tardis-machine HTTP API running on localhost:8000 const url = `http://localhost:8000/replay-normalized?options=${serialize( options )}` const response = await fetch(url) // returned data is in NDJSON format http://ndjson.org/ // each line is separate message JSON encoded // split response body stream by new lines const lines = Readable.fromWeb(response.body).pipe(split2()) for await (const line of lines) { yield line } } const options = { exchange: 'binance', from: '2024-03-01', to: '2024-03-02', symbols: ['btcusdt'], withDisconnectMessages: true, // other available data types examples: // 'book_snapshot_10_100ms', 'derivative_ticker', 'quote', // 'trade_bar_10ms', 'trade_bar_10s' dataTypes: ['trade', 'book_change', 'book_snapshot_10_100ms'] } const lines = replayNormalizedViaTardisMachine(options) for await (const line of lines) { const normalizedMessage = JSON.parse(line) console.log(normalizedMessage) } ``` {% hint style="info" %} See also official Tardis.dev [Node.js client](https://docs.tardis.dev/node-client/quickstart) library. {% endhint %} {% endtab %} {% tab title="cURL" %} ```bash curl -g 'http://localhost:8000/replay-normalized?options={"exchange":"binance","from":"2024-03-01","to":"2024-03-02","symbols":["btcusdt"],"withDisconnectMessages":true,"dataTypes":["trade","book_change","book_snapshot_10_100ms"]}' ``` {% embed url="" %} Click to see API response in the browser as long as tardis-machine is running on localhost:8000 {% endembed %} {% endtab %} {% tab title="Your preferred language" %} We're working on providing more samples and dedicated client libraries in different languages, but in the meanwhile to consume [HTTP /replay-normalized](#http-get-replay-normalized-options-options) API responses in your language of choice, you should: 1. Provide URL-encoded JSON [options](#replay-normalized-options) via the options query string parameter when sending an HTTP request 2. Parse HTTP response stream line by line as it's returned - buffering in memory whole response may result in slow performance and memory overflows 3. Parse each response line as JSON containing [normalized data messages](https://docs.tardis.dev/tardis-machine/data-types). {% endtab %} {% endtabs %} #### Replay normalized options [HTTP /replay-normalized](#http-get-replay-normalized-options-options) endpoint accepts required options query string param in **url encoded JSON format**. Options JSON needs to be an object or an array of objects with fields as specified below. If array is provided, then data requested for multiple exchanges is returned synchronized (by local timestamp). **Synchronized multi-exchange example** The same `options` array format works for both [`HTTP /replay-normalized`](#http-get-replay-normalized-options-options) and [`WebSocket /ws-replay-normalized`](#websocket-ws-replay-normalized-options-options). For example, to replay synchronized normalized trades from Binance and OKX for the same symbol and time range: ```bash curl -g 'http://localhost:8000/replay-normalized?options=[{"exchange":"binance","from":"2024-03-01","to":"2024-03-02","symbols":["btcusdt"],"dataTypes":["trade"]},{"exchange":"okex","from":"2024-03-01","to":"2024-03-02","symbols":["btcusdt"],"dataTypes":["trade"]}]' ``` Each response line is still a single normalized message, but messages from both exchanges are merged into one stream and ordered by `localTimestamp`. | name | type | default | description | | ------------------------------- | ---------------------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **exchange** | string | - | requested exchange id - use [/exchanges HTTP API](https://docs.tardis.dev/api/http-api-reference#exchanges) to get list of valid exchanges ids | | **symbols** | string\[] (optional) | undefined | optional symbols of requested historical data feed - use [/exchanges/:exchange HTTP API](https://docs.tardis.dev/api/http-api-reference#exchanges-exchange) to get allowed symbols for requested exchange | | **from** | string | - | replay period start date (UTC) in a [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format, e.g., `2019-04-01` | | **to** | string | - | replay period end date (UTC) in a [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format, e.g., `2019-04-02` | | **dataTypes** | string\[] | - | array of normalized [data types](https://docs.tardis.dev/tardis-machine/data-types) for which historical data will be returned | | **withDisconnectMessages** | boolean (optional) | undefined | when set to `true`, response includes [disconnect](https://docs.tardis.dev/data-types#disconnect) messages that mark events when real-time WebSocket connection that was used to collect the historical data got disconnected | | **waitWhenDataNotYetAvailable** | boolean or number (optional) | undefined | when set to `true`, waits for data that is not yet available — useful when replaying near real-time data. Defaults to a 30-minute offset. When set to a number, specifies the offset in minutes (minimum effective value is 6 minutes). | | **autoCleanup** | boolean (optional) | undefined | when set to `true`, automatically removes cached data from disk after it has been processed. Not safe for concurrent replay — see [warning](#replay-options) | #### **Response format & sample messages** See [Output Data Types](https://docs.tardis.dev/tardis-machine/data-types). ### `WebSocket` /ws-replay-normalized?options={options} Sends [normalized](https://docs.tardis.dev/tardis-machine/data-types) historical market data for [data types](#replay-normalized-options-1) specified via query string. See [supported data types](https://docs.tardis.dev/tardis-machine/data-types) which include normalized [trade](https://docs.tardis.dev/data-types#trade), [order book change](https://docs.tardis.dev/data-types#book_change), [customizable order book snapshots](https://docs.tardis.dev/data-types#book_snapshot_-number_of_levels-_-snapshot_interval-time_unit) etc. {% hint style="info" %} [WebSocket /ws-stream-normalized](https://docs.tardis.dev/streaming-real-time-data#websocket-ws-stream-normalized-options-options) is the real-time counterpart of this API endpoint, providing real-time market data in the same format, but not requiring API key as connects directly to exchanges' real-time WebSocket APIs. {% endhint %} {% tabs %} {% tab title="Python" %} ```python import asyncio import aiohttp import json import urllib.parse async def main(): replay_options = { "exchange": "binance", "from": "2024-03-01", "to": "2024-03-02", "symbols": ["btcusdt"], "withDisconnectMessages": True, # other available data types examples: # 'book_snapshot_10_100ms', 'derivative_ticker', 'quote', # 'trade_bar_10ms', 'trade_bar_10s' "dataTypes": ["trade", "book_change", "book_snapshot_10_100ms"], } options = urllib.parse.quote_plus(json.dumps(replay_options)) URL = f"ws://localhost:8001/ws-replay-normalized?options={options}" async with aiohttp.ClientSession() as session: async with session.ws_connect(URL) as websocket: async for msg in websocket: print(msg.data) asyncio.run(main()) ``` {% endtab %} {% tab title="Node.js" %} ```javascript import WebSocket from 'ws' const serialize = options => { return encodeURIComponent(JSON.stringify(options)) } const replayOptions = { exchange: 'binance', from: '2024-03-01', to: '2024-03-02', symbols: ['btcusdt'], withDisconnectMessages: true, // other available data types examples: // 'book_snapshot_10_100ms', 'derivative_ticker', 'quote', // 'trade_bar_10ms', 'trade_bar_10s' dataTypes: ['trade', 'book_change', 'book_snapshot_10_100ms'] } const options = serialize(replayOptions) const URL = `ws://localhost:8001/ws-replay-normalized?options=${options}` const ws = new WebSocket(URL) ws.onmessage = message => { console.log(message.data) } ``` {% hint style="info" %} See also official Tardis.dev [Node.js client](https://docs.tardis.dev/node-client/quickstart) library. {% endhint %} {% endtab %} {% tab title="Your preferred language" %} We're working on providing more samples and dedicated client libraries in different languages, but in the meanwhile to consume [WebSocket /ws-replay-normalized](#websocket-ws-replay-normalized-options-options) API responses in your language of choice, you should: 1. Provide url encoded JSON [options](#replay-normalized-options-1) via options query string param when connecting to [WebSocket /ws-replay-normalized](#websocket-ws-replay-normalized-options-options) endpoint 2. Parse each received WebSocket message as JSON containing [normalized data](https://docs.tardis.dev/tardis-machine/data-types). {% endtab %} {% endtabs %} #### Replay normalized options [WebSocket /ws-replay-normalized](#websocket-ws-replay-normalized-options-options) endpoint accepts required options query string param in **url encoded JSON format**. Options JSON needs to be an object or an array of objects with fields as specified below. If array is provided, then data requested for multiple exchanges is sent synchronized (by local timestamp). See the [multi-exchange example above](#replay-normalized-options). | name | type | default | description | | ------------------------------- | ---------------------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **exchange** | string | - | requested exchange id - use [/exchanges HTTP API](https://docs.tardis.dev/api/http-api-reference#exchanges) to get list of valid exchanges ids | | **symbols** | string\[] (optional) | undefined | optional symbols of requested historical data feed - use [/exchanges/:exchange HTTP API](https://docs.tardis.dev/api/http-api-reference#exchanges-exchange) to get allowed symbols for requested exchange | | **from** | string | - | replay period start date (UTC) in a [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format, e.g., `2019-04-01` | | **to** | string | - | replay period end date (UTC) in a [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format, e.g., `2019-04-02` | | **dataTypes** | string\[] | - | array of normalized [data types](https://docs.tardis.dev/tardis-machine/data-types) for which historical data will be provided | | **withDisconnectMessages** | boolean (optional) | undefined | when set to `true`, sends also [disconnect](https://docs.tardis.dev/data-types#disconnect) messages that mark events when real-time WebSocket connection that was used to collect the historical data got disconnected | | **waitWhenDataNotYetAvailable** | boolean or number (optional) | undefined | when set to `true`, waits for data that is not yet available — useful when replaying near real-time data. Defaults to a 30-minute offset. When set to a number, specifies the offset in minutes (minimum effective value is 6 minutes). | | **autoCleanup** | boolean (optional) | undefined | when set to `true`, automatically removes cached data from disk after it has been processed. Not safe for concurrent replay — see [warning](#replay-options) | {% hint style="info" %} In our preliminary benchmarks on AMD Ryzen 7 3700X, 64GB RAM, [WebSocket /ws-replay-normalized](#websocket-ws-replay-normalized-options-options) API endpoint was returning \~70 000 messages/s and \~40 000 messages/s when order book snapshots were also requested. {% endhint %} #### **Response format & sample messages** See [Output Data Types](https://docs.tardis.dev/tardis-machine/data-types). --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.tardis.dev/tardis-machine/replaying-historical-data.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.