Tardis.dev Documentation
  • Welcome
  • Frequently Asked Questions
    • General
    • Data
    • Billing and Subscriptions
  • Downloadable CSV files
  • Historical Data Details
    • BitMEX
    • Deribit
    • Binance USDT Futures
    • Binance COIN Futures
    • Binance Spot
    • FTX
    • OKX Futures
    • OKX Swap
    • OKX Options
    • OKX Spot
    • Huobi Futures
    • Huobi COIN Swaps
    • Huobi USDT Swaps
    • Huobi Global
    • Bitfinex Derivatives
    • Bitfinex
    • Coinbase Pro
    • Kraken Futures
    • Kraken
    • Bitstamp
    • Gemini
    • Bybit Derivatives
    • Bybit Spot
    • dYdX
    • WOO X
    • Kucoin Spot
    • Blockchain.com
    • Upbit
    • Phemex
    • Delta
    • AscendEX (BitMax)
    • FTX US
    • Binance US
    • Gate.io Futures
    • Gate.io
    • Bitnomial
    • Crypto.com
    • OKCoin
    • bitFlyer
    • HitBTC (high caps)
    • CoinFLEX (2.0)
    • Binance Jersey
    • Binance DEX
    • Poloniex
  • API
    • Getting Started
    • Python Client
    • Node.js Client
    • Tardis Machine Server
    • Instruments Metadata API
    • HTTP API Reference
  • Legal
    • Privacy Policy
    • Terms of Service
  • External Links
    • Public Roadmap
    • Open Source
    • Newsletter
    • Contact Us
    • Get API Key
    • Tardis.dev Home
Powered by GitBook
On this page
  • Introduction
  • Features
  • Installation
  • Docker
  • npm
  • Exchange-native market data APIs
  • • HTTP GET /replay?options={options}
  • • WebSocket /ws-replay?exchange={exchange}&from={fromDate}&to={toDate}
  • Normalized market data APIs
  • • HTTP GET /replay-normalized?options={options}
  • • WebSocket /ws-replay-normalized?options={options}
  • • WebSocket /ws-stream-normalized?options={options}
  • Normalized data types
  • • trade
  • • book_change
  • • derivative_ticker
  • • book_snapshot_{number_of_levels}_{snapshot_interval}{time_unit}
  • • trade_bar_{aggregation_interval}{suffix}
  • • disconnect
Export as PDF
  1. API

Tardis Machine Server

Locally runnable server with built-in data caching, providing both tick-level historical and consolidated real-time cryptocurrency market data via HTTP and WebSocket APIs

PreviousNode.js ClientNextInstruments Metadata API

Last updated 1 year ago

Introduction

is a locally runnable server with built-in data caching that uses Tardis.dev under the hood. It provides both tick-level historical and consolidated real-time cryptocurrency market data via it's HTTP and WebSocket APIs and is available via and .

Features

    • consistent format for accessing market data across multiple exchanges

  • transparent historical local data caching (cached data is stored on disk in compressed GZIP format and decompressed on demand when reading the data)

Installation

Docker

### running without persitent local cache
docker run -p 8000:8000 -p 8001:8001 -e "TM_API_KEY=YOUR_API_KEY" -d tardisdev/tardis-machine

Tardis-machine server's HTTP endpoints will be available on port 8000 and WebSocket API endpoints on port 8001. Your API key will be passed via ENV variable (TM_API_KEY) — simply replace YOUR_API_KEY with API key you've received via email.

docker run -v ./host-cache-dir:/.cache -p 8000:8000 -p 8001:8001 -e "TM_API_KEY=YOUR_API_KEY" -d tardisdev/tardis-machine

Since using volumes can cause issues especially on Windows, it's fine to run Docker image without them with the caveat of potentially poor local cache ratio after each container's restart.

Config environment variables

You can set following environment config variables to configure tardis-machine server:

name

default

description

TM_API_KEY

TM_PORT

8000

HTTP port on which server will be running, WebSocket port is always this value + 1 (8001 with port set to 8000)

TM_CACHE_DIR

/.cache

path to local dir that will be used as cache location

TM_CLUSTER_MODE

false

TM_DEBUG

false

server will print verbose debug logs to stdout if set to true

TM_CLEAR_CACHE

false

server will clear local cache dir on startup if set to true

npm

Requires Node.js v12+ and git installed.

Install and runtardis-machine server via npx command:

npx tardis-machine --api-key=YOUR_API_KEY

or install globally via npm:

npm install -g tardis-machine

and then run:

tardis-machine --api-key=YOUR_API_KEY

Tardis-machine server's HTTP endpoints will be available on port 8000 and WebSocket API endpoints on port 8001. Your API key will be passed via --api-key config flag — simply replace YOUR_API_KEY with API key you've received via email.

CLI config flags

You can set following CLI config flags when starting tardis-machine server installed via npm:

name

default

description

--api-key

--port

8000

HTTP port on which server will be running, WebSocket port is always this value + 1 (8001 with port set to 8000)

--cache-dir

<os.tmpdir>/.tardis-cache

path to local dir that will be used as cache location - if not provided default temp dir for given OS will be used

--cluster-mode

false

--debug

false

server will print verbose debug logs to stdout if set to true

--clear-cache

false

server will clear local cache dir on startup is set to true

--help

shows CLI help

--version

shows tardis-machine version number

Exchange-native market data APIs

• HTTP GET /replay?options={options}

import asyncio
import aiohttp
import json
import urllib.parse


async def replay_via_tardis_machine_machine(replay_options):
    timeout = aiohttp.ClientTimeout(total=0)

    async with aiohttp.ClientSession(timeout=timeout) as session:
        # url encode as json object options
        encoded_options = urllib.parse.quote_plus(json.dumps(replay_options))

        # assumes tardis-machine HTTP API running on localhost:8000
        url = f"http://localhost:8000/replay?options={encoded_options}"

        async with session.get(url) as response:
            # otherwise we may get line to long errors
            response.content._high_water = 100_000_000

            # returned data is in NDJSON format http://ndjson.org/
            # each line is separate message JSON encoded
            async for line in response.content:
                yield line


async def run():
    lines = replay_via_tardis_machine_machine(
        {
            "exchange": "bitmex",
            "from": "2019-10-01",
            "to": "2019-10-02",
            "filters": [
                {"channel": "trade", "symbols": ["XBTUSD", "ETHUSD"]},
                {"channel": "orderBookL2", "symbols": ["XBTUSD", "ETHUSD"]},
            ],
        }
    )

    async for line in lines:
        message = json.loads(line)
        # localTimestamp string marks timestamp when message was received
        # message is a message dict as provided by exchange real-time stream
        print(message["localTimestamp"], message["message"])


asyncio.run(run())
const fetch = require('node-fetch')
const split2 = require('split2')


const serialize = options => {
  return encodeURIComponent(JSON.stringify(options))
}

async function* replayViaTardisMachine(options) {
  // assumes tardis-machine HTTP API running on localhost:8000
  const url = `http://localhost:8000/replay?options=${serialize(options)}`
  const response = await fetch(url)

  // returned data is in NDJSON format http://ndjson.org/
  // each line is separate message JSON encoded

  // split response body stream by new lines
  const lines = response.body.pipe(split2())

  for await (const line of lines) {
    yield line
  }
}

async function run() {
  const options = {
    exchange: 'bitmex',
    from: '2019-10-01',
    to: '2019-10-02',
    filters: [
      {
        channel: 'trade',
        symbols: ['XBTUSD', 'ETHUSD']
      },
      {
        channel: 'orderBookL2',
        symbols: ['XBTUSD', 'ETHUSD']
      }
    ]
  }

  const lines = replayViaTardisMachine(options)

  for await (const line of lines) {
    // localTimestamp string marks timestamp when message was received
    // message is a message object as provided by exchange real-time stream
    const { message, localTimestamp } = JSON.parse(line)

    console.log(message, localTimestamp)
  }
}

run()
curl -g 'localhost:8000/replay?options={"exchange":"bitmex","filters":[{"channel":"orderBookL2","symbols":["XBTUSD","ETHUSD"]}],"from":"2019-07-01","to":"2019-07-02"}'
  1. Parse HTTP response stream line by line as it's returned - buffering in memory whole response may result in slow performance and memory overflows

Replay options

name

type

default

exchange

string

-

filters

{channel:string, symbols?: string[]}[]

[]

from

string

-

to

string

-

withDisconnects

boolean | undefined

undefined

when set to true, response includes empty lines (\n) that mark events when real-time WebSocket connection that was used to collect the historical data got disconnected

Response format

  • localTimestamp - date when message has been received in ISO 8601 format

  • message - JSON with exactly the same format as provided by requested exchange real-time feeds

Sample response

{"localTimestamp":"2019-05-01T00:09:42.2760012Z","message":{"table":"orderBookL2","action":"update","data":[{"symbol":"XBTUSD","id":8799473750,"side":"Buy","size":2333935}]}}
{"localTimestamp":"2019-05-01T00:09:42.2932826Z","message":{"table":"orderBookL2","action":"update","data":[{"symbol":"XBTUSD","id":8799474250,"side":"Buy","size":227485}]}}
{"localTimestamp":"2019-05-01T00:09:42.4249304Z","message":{"table":"trade","action":"insert","data":[{"timestamp":"2019-05-01T00:09:42.407Z","symbol":"XBTUSD","side":"Buy","size":1500,"price":5263,"tickDirection":"ZeroPlusTick","trdMatchID":"29d7de7f-27b6-9574-48d1-3ee9874831cc","grossValue":28501500,"homeNotional":0.285015,"foreignNotional":1500}]}}
{"localTimestamp":"2019-05-01T00:09:42.4249403Z","message":{"table":"orderBookL2","action":"update","data":[{"symbol":"XBTUSD","id":8799473700,"side":"Sell","size":454261}]}}
{"localTimestamp":"2019-05-01T00:09:42.4583155Z","message":{"table":"orderBookL2","action":"update","data":[{"symbol":"XBTUSD","id":8799473750,"side":"Buy","size":2333838},{"symbol":"XBTUSD","id":8799473800,"side":"Buy","size":547746}]}}

• WebSocket /ws-replay?exchange={exchange}&from={fromDate}&to={toDate}

After connection is established, client has 2 seconds to send subscriptions payloads and then market data replay starts.

import asyncio
import aiohttp
import json


async def run():
    WS_REPLAY_URL = "ws://localhost:8001/ws-replay"
    URL = f"{WS_REPLAY_URL}?exchange=bitmex&from=2019-10-01&to=2019-10-02"

    async with aiohttp.ClientSession() as session:
        async with session.ws_connect(URL) as websocket:

            await websocket.send_str(
                json.dumps(
                    {
                        "op": "subscribe",
                        "args": [
                            "trade:XBTUSD",
                            "trade:ETHUSD",
                            "orderBookL2:XBTUSD",
                            "orderBookL2:ETHUSD",
                        ],
                    }
                )
            )

            async for msg in websocket:
                print(msg.data)


asyncio.run(run())

You can also try using existing WebSocket client by changing URL endpoint to the one shown in the example above.

const WebSocket = require('ws')


const WS_REPLAY_URL = 'ws://localhost:8001/ws-replay'

const ws = new WebSocket(
  `${WS_REPLAY_URL}?exchange=bitmex&from=2019-10-01&to=2019-10-02`
)

ws.onmessage = message => {
  console.log(message.data)
}

ws.onopen = () => {
  ws.send(
    JSON.stringify({
      op: 'subscribe',
      args: [
        'trade:XBTUSD',
        'trade:ETHUSD',
        'orderBookL2:XBTUSD',
        'orderBookL2:ETHUSD'
      ]
    })
  )
}
const ccxws = require('ccxws')
const BASE_URL = 'ws://localhost:8001/ws-replay'
const WS_REPLAY_URL = `${BASE_URL}?exchange=bitmex&from=2019-10-01&to=2019-10-02`

const bitMEXClient = new ccxws.bitmex()
// only change required for ccxws client is to point it to /ws-replay URL
bitMEXClient._wssPath = WS_REPLAY_URL

const market = {
  id: 'XBTUSD',
  base: 'BTC',
  quote: 'USD'
}

bitMEXClient.on('l2snapshot', snapshot =>
  console.log('snapshot', snapshot.asks.length, snapshot.bids.length)
)

bitMEXClient.on('l2update', update => console.log(update))

bitMEXClient.on('trade', trade => console.log(trade))

bitMEXClient.subscribeTrades(market)
bitMEXClient.subscribeLevel2Updates(market)

Query string params

name

type

default

description

exchange

string

-

from

string

-

to

string

-

session

string | undefined

undefined

optional replay session key. When specified and multiple clients use it when connecting at the same time then data being send to those clients is synchronized (by local timestamp).

Normalized market data APIs

• HTTP GET /replay-normalized?options={options}

import asyncio
import aiohttp
import json
import urllib.parse


async def replay_normalized_via_tardis_machine_machine(replay_options):
    timeout = aiohttp.ClientTimeout(total=0)

    async with aiohttp.ClientSession(timeout=timeout) as session:
        # url encode as json object options
        encoded_options = urllib.parse.quote_plus(json.dumps(replay_options))

        # assumes tardis-machine HTTP API running on localhost:8000
        url = f"http://localhost:8000/replay-normalized?options={encoded_options}"

        async with session.get(url) as response:
            # otherwise we may get line to long errors
            response.content._high_water = 100_000_000

            # returned data is in NDJSON format http://ndjson.org/ streamed
            # each line is separate message JSON encoded
            async for line in response.content:
                yield line


async def run():
    lines = replay_normalized_via_tardis_machine_machine(
        {
            "exchange": "bitmex",
            "from": "2019-10-01",
            "to": "2019-10-02",
            "symbols": ["XBTUSD", "ETHUSD"],
            "withDisconnectMessages": True,
            # other available data types examples:
            # 'book_snapshot_10_100ms', 'derivative_ticker', 'quote',
            # 'trade_bar_10ms', 'trade_bar_10s'
            "dataTypes": ["trade", "book_change", "book_snapshot_10_100ms"],
        }
    )

    async for line in lines:
        normalized_message = json.loads(line)
        print(normalized_message)


asyncio.run(run())
const fetch = require('node-fetch')
const split2 = require('split2')


const serialize = options => {
  return encodeURIComponent(JSON.stringify(options))
}

async function* replayNormalizedViaTardisMachine(options) {
  // assumes tardis-machine HTTP API running on localhost:8000
  const url = `http://localhost:8000/replay-normalized?options=${serialize(
    options
  )}`
  const response = await fetch(url)

  // returned data is in NDJSON format http://ndjson.org/
  // each line is separate message JSON encoded

  // split response body stream by new lines
  const lines = response.body.pipe(split2())

  for await (const line of lines) {
    yield line
  }
}

async function run() {
  const options = {
    exchange: 'bitmex',
    from: '2019-10-01',
    to: '2019-10-02',
    symbols: ['XBTUSD', 'ETHUSD'],
    withDisconnectMessages: true,
    // other available data types examples:
    // 'book_snapshot_10_100ms', 'derivative_ticker', 'quote',
    // 'trade_bar_10ms', 'trade_bar_10s'
    dataTypes: ['trade', 'book_change', 'book_snapshot_10_100ms']
  }

  const lines = replayNormalizedViaTardisMachine(options)

  for await (const line of lines) {
    const normalizedMessage = JSON.parse(line)

    console.log(normalizedMessage)
  }
}

run()
curl -g 'http://localhost:8000/replay-normalized?options={"exchange":"bitmex","from":"2019-10-01","to":"2019-10-02","symbols":["XBTUSD","ETHUSD"],"withDisconnectMessages":true,"dataTypes":["trade","book_change","book_snapshot_10_100ms"]}'
  1. Parse HTTP response stream line by line as it's returned - buffering in memory whole response may result in slow performance and memory overflows

Replay normalized options

Options JSON needs to be an object or an array of objects with fields as specified below. If array is provided, then data requested for multiple exchanges is returned synchronized (by local timestamp).

name

type

default

exchange

string

-

symbols

string[] | undefined

undefined

from

string

-

to

string

-

dataTypes

string[]

-

withDisconnectMessages

boolean | undefined

undefined

Response format & sample messages

• WebSocket /ws-replay-normalized?options={options}

import asyncio
import aiohttp
import json
import urllib.parse


async def run():
    replay_options = {
        "exchange": "bitmex",
        "from": "2019-10-01",
        "to": "2019-10-02",
        "symbols": ["XBTUSD", "ETHUSD"],
        "withDisconnectMessages": True,
        # other available data types examples:
        # 'book_snapshot_10_100ms', 'derivative_ticker', 'quote',
        # 'trade_bar_10ms', 'trade_bar_10s'
        "dataTypes": ["trade", "book_change", "book_snapshot_10_100ms"],
    }

    options = urllib.parse.quote_plus(json.dumps(replay_options))

    URL = f"ws://localhost:8001/ws-replay-normalized?options={options}"

    async with aiohttp.ClientSession() as session:
        async with session.ws_connect(URL) as websocket:
            async for msg in websocket:
                print(msg.data)


asyncio.run(run())
const WebSocket = require('ws')


const serialize = options => {
  return encodeURIComponent(JSON.stringify(options))
}

const replayOptions = {
  exchange: 'bitmex',
  from: '2019-10-01',
  to: '2019-10-02',
  symbols: ['XBTUSD', 'ETHUSD'],
  withDisconnectMessages: true,
  // other available data types examples:
  // 'book_snapshot_10_100ms', 'derivative_ticker', 'quote',
  // 'trade_bar_10ms', 'trade_bar_10s'
  dataTypes: ['trade', 'book_change', 'book_snapshot_10_100ms']
}

const options = serialize(replayOptions)
const URL = `ws://localhost:8001/ws-replay-normalized?options=${options}`
const ws = new WebSocket(URL)

ws.onmessage = message => {
  console.log(message.data)
}

Replay normalized options

Options JSON needs to be an object or an array of objects with fields as specified below. If array is provided, then data requested for multiple exchanges is being send synchronized (by local timestamp).

name

type

default

exchange

string

-

symbols

string[] | undefined

undefined

from

string

-

to

string

-

dataTypes

string[]

-

withDisconnectMessages

boolean | undefined

undefined

Response format & sample messages

• WebSocket /ws-stream-normalized?options={options}

Doesn't requires API key as connects directly to exchanges real-time WebSocket APIs and transparently restarts closed, broken or stale connections (open connections without data being send for specified amount of time).

import asyncio
import aiohttp
import json
import urllib.parse


async def run():
    data_types = ["trade", "book_change", "book_snapshot_10_100ms"]

    stream_options = [
        {
            "exchange": "bitmex",
            "symbols": ["XBTUSD"],
            "dataTypes": data_types,
        },
        {
            "exchange": "deribit",
            "symbols": ["BTC-PERPETUAL"],
            "dataTypes": data_types,
        },
    ]

    options = urllib.parse.quote_plus(json.dumps(stream_options))

    URL = f"ws://localhost:8001/ws-stream-normalized?options={options}"
    # real-time normalized data for two exchanges via single connection
    async with aiohttp.ClientSession() as session:
        async with session.ws_connect(URL) as websocket:

            async for msg in websocket:
                print(msg.data)


asyncio.run(run())
const WebSocket = require('ws')


const serialize = options => {
  return encodeURIComponent(JSON.stringify(options))
}
// other available data types examples:
// 'book_snapshot_10_100ms', 'derivative_ticker', 'quote',
// 'trade_bar_10ms', 'trade_bar_10s'
const dataTypes = ['trade', 'book_change', 'book_snapshot_10_100ms']

const streamOptions = [
  {
    exchange: 'bitmex',
    symbols: ['XBTUSD'],
    dataTypes
  },
  {
    exchange: 'deribit',
    symbols: ['BTC-PERPETUAL'],
    dataTypes
  }
]

const options = serialize(streamOptions)
const URL = `ws://localhost:8001/ws-stream-normalized?options=${options}`
const ws = new WebSocket(URL)
// real-time normalized data for two exchanges via single connection
ws.onmessage = message => {
  console.log(message.data)
}

Stream normalized options

Options JSON needs to be an object or an array of objects with fields as specified below. If array is specified then API provides single consolidated real-time data stream for all exchanges specified (as in examples above).

name

type

default

exchange

string

-

symbols

string[] | undefined

undefined

optional symbols of requested real-time data feed

dataTypes

string[]

-

withDisconnectMessages

boolean | undefined

undefined

timeoutIntervalMS

number

10000

specifies time in milliseconds after which connection to real-time exchanges' WebSocket API is restarted if no message has been received

Response format & sample messages

Normalized data types

• trade

Individual trade

{
  "type": "trade",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "id": "282a0445-0e3a-abeb-f403-11003204ea1b",
  "price": 7996,
  "amount": 50,
  "side": "sell",
  "timestamp": "2019-10-23T10:32:49.669Z",
  "localTimestamp": "2019-10-23T10:32:49.740Z"
}
{
  type: 'trade'
  symbol: string // instrument symbol as provided by exchange
  exchange: string // exchange id
  id: string | undefined // trade id if provided by exchange
  price: number // trade price as provided by exchange
  amount: number // trade amount as provided by exchange
  side: 'buy' | 'sell' | 'unknown' // liquidity taker side (aggressor)
  timestamp: string // trade timestamp provided by exchange (ISO 8601 format)
  localTimestamp: string // message arrival timestamp (ISO 8601 format)
}

• book_change

{
  "type": "book_change",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "isSnapshot": false,
  "bids": [],
  "asks": [
    {
      "price": 7985,
      "amount": 283318
    }
  ],
  "timestamp": "2019-10-23T11:29:53.469Z",
  "localTimestamp": "2019-10-23T11:29:53.469Z"
}
{
  type: 'book_change'
  symbol: string // instrument symbol as provided by exchange
  exchange: string // exchange id
  isSnapshot: boolean // if true marks initial order book snapshot
  bids: { price: number; amount: number }[] // updated bids price-amount levels
  asks: { price: number; amount: number }[] // updated asks price-amount levels
  timestamp: string // order book update timestamp if provided by exchange,
                    // otherwise equals to localTimestamp, (ISO 8601 format)
  localTimestamp: string // message arrival timestamp (ISO 8601 format)
}

• derivative_ticker

Derivative instrument ticker info sourced from real-time ticker & instrument channels.

{
  "type": "derivative_ticker",
  "symbol": "BTC-PERPETUAL",
  "exchange": "deribit",
  "lastPrice": 7987.5,
  "openInterest": 84129491,
  "fundingRate": -0.00001568,
  "indexPrice": 7989.28,
  "markPrice": 7987.56,
  "timestamp": "2019-10-23T11:34:29.302Z",
  "localTimestamp": "2019-10-23T11:34:29.416Z"
}
{
  type: 'derivative_ticker'
  symbol: string // instrument symbol as provided by exchange
  exchange: string // exchange id
  lastPrice: number | undefined // last instrument price if provided by exchange
  openInterest: number | undefined // last open interest if provided by exchange
  fundingRate: number | undefined // last funding rate if provided by exchange
  indexPrice: number | undefined // last index price if provided by exchnage
  markPrice: number | undefined // last mark price if provided by exchange
  timestamp: string // message timestamp provided by exchange (ISO 8601 format)
  localTimestamp: string // message arrival timestamp (ISO 8601 format)
}

• book_snapshot_{number_of_levels}_{snapshot_interval}{time_unit}

Examples:

  • book_snapshot_10_0ms - provides top 10 levels tick-by-tick order book snapshots

  • book_snapshot_50_100ms - provides top 50 levels order book snapshots taken at 100 millisecond intervals

  • book_snapshot_30_10s - provides top 30 levels order book snapshots taken at 10 second intervals

  • quote is an alias of book_snapshot_1_0ms - provides top of the book (best bid/ask) tick-by-order book snapshots

  • quote_10s is an alias of book_snapshot_1_10s - provides top of the book (best bid/ask) order book snapshots taken at 10 seconds intervals

Available time units:

  • ms - milliseconds

  • s - seconds

  • m - minutes

{
  "type": "book_snapshot",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "name": "book_snapshot_2_50ms",
  "depth": 2,
  "interval": 50,
  "bids": [
    {
      "price": 7633.5,
      "amount": 1906067
    },
    {
      "price": 7633,
      "amount": 65319
    }
  ],
  "asks": [
    {
      "price": 7634,
      "amount": 1467849
    },
    {
      "price": 7634.5,
      "amount": 67939
    }
  ],
  "timestamp": "2019-10-25T13:39:46.950Z",
  "localTimestamp": "2019-10-25T13:39:46.961Z"
}
{
  type: 'book_snapshot'
  symbol: string // instrument symbol as provided by exchange
  exchange: string // exchange id
  name: string // name with format book_snapshot_{depth}_{interval}{time_unit}
  depth: number // requested number of levels (top bids/asks)
  interval: number // requested snapshot interval in milliseconds
  bids: { price: number; amount: number }[] // top "depth" bids price-amount levels
  asks: { price: number; amount: number }[] // top "depth" asks price-amount levels
  timestamp: string // snapshot timestamp based on last book_change message
                    // processed timestamp adjusted to snapshot interval
  localTimestamp: string // message arrival timestamp 
                         // that triggered snapshot (ISO 8601 format)
}

• trade_bar_{aggregation_interval}{suffix}

Trades data in aggregated form, known as OHLC, candlesticks, klines etc. Not only most common time based aggregation is supported, but volume and tick count based as well. Bars are computed from tick-by-tick raw trade data, if in given interval no trades happened, there is no bar produced.

Examples:

  • trade_bar_10ms - provides time based trade bars with 10 milliseconds intervals

  • trade_bar_5m - provides time based trade bars with 5 minute intervals

  • trade_bar_100ticks - provides ticks based trade bars with 100 ticks (individual trades) intervals

  • trade_bar_100000vol - provides volume based trade bars with 100 000 volume intervals

Allowed suffixes:

  • ms - milliseconds

  • s - seconds

  • m - minutes

  • ticks - number of ticks

  • vol - volume size

{
  "type": "trade_bar",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "name": "trade_bar_10000ms",
  "interval": 10000,
  "kind": "time",
  "open": 7623.5,
  "high": 7623.5,
  "low": 7623,
  "close": 7623.5,
  "volume": 37034,
  "buyVolume": 24244,
  "sellVolume": 12790,
  "trades": 9,
  "vwap": 7623.327320840309,
  "openTimestamp": "2019-10-25T13:11:31.574Z",
  "closeTimestamp": "2019-10-25T13:11:39.212Z",
  "localTimestamp": "2019-10-25T13:11:40.369Z",
  "timestamp": "2019-10-25T13:11:40.000Z"
}
{
  type: 'trade_bar'
  symbol: string // instrument symbol as provided by exchange
  exchange: string // exchange id
  name: string // name with format trade_bar_{interval}
  interval: number // requested trade bar interval
  kind: 'time' | 'volume' | 'tick' // trade bar kind
  open: number // open price
  high: number // high price
  low: number //low price
  close: number // close price
  volume: number // total volume traded in given interval
  buyVolume: number // buy volume traded in given interval
  sellVolume: number // sell volume traded in given interval
  trades: number // trades count in given interval
  vwap: number // volume weighted average price
  openTimestamp: string // timestamp of first trade for given bar (ISO 8601 format)
  closeTimestamp: string // timestamp of last trade for given bar (ISO 8601 format)
  timestamp: string // end of interval period timestamp (ISO 8601 format)
  localTimestamp: string // message arrival timestamp 
                         // that triggered given bar computation (ISO 8601 format)
}

• disconnect

Message that marks events when real-time WebSocket connection that was used to collect the historical data got disconnected.

{
  "type": "disconnect",
  "exchange": "deribit",
  "localTimestamp": "2019-10-23T11:34:29.416Z"
}
{
  type: 'disconnect'
  exchange: string // exchange id
  localTimestamp: string // timestamp when disconnect occurred (ISO 8601 format)
}

efficient data replay API endpoints returning historical market data for whole time periods (in contrast to Tardis.dev where single call returns data for single minute time period)

tick-by-tick historical market data replay in

and endpoints

providing historical market data replay from any given past point in time with the same data format and 'subscribe' logic as real-time exchanges' APIs - in many cases existing exchanges' WebSocket clients can be used to connect to this endpoint

and endpoints

synchronized

connecting directly to exchanges' WebSocket APIs

customizable and data types

seamless

support for top cryptocurrency exchanges: , , , , , , , , , , , , , , , , , and more

Pull and run latest version of :

Command above does not use persistent volumes for local caching (each docker restart will result in loosing local data cache). In order to use for example./host-cache-dir as persistent volume () cache directory, run:

API key for HTTP API - if not provided only first day of each month of historical data is accessible

will launch to handle the incoming requests if set to true, by default server runs in single process mode

You can configure tardis-machine server via as described in Docker section as well.

API key for HTTP API - if not provided only first day of each month of historical data is accessible

will launch to handle the incoming requests if set to true, by default server runs in single process mode

Exchange-native market data API endpoints provide historical data in . The main difference between HTTP and WebSocket endpoints is the logic of requesting data:

accepts payload via query string param

accepts exchanges' specific 'subscribe' messages that define what data will be then "replayed" and send to WebSocket client

Returns historical market data messages in for given replay options query string param. Single streaming HTTP response returns data for the whole requested time period as .

In our preliminary benchmarks on AMD Ryzen 7 3700X, 64GB RAM, API endpoint was returning ~700 000 messages/s (already locally cached data).

See also official Tardis.dev library.

See also official Tardis.dev library.

We're working on providing more samples and dedicated client libraries in different languages, but in the meanwhile to consume API responses in your language of choice, you should:

Provide url encoded JSON via options query string param when sending HTTP request

Parse each response line as JSON containing messages in

endpoint accepts required options query string param in url encoded JSON format.

requested exchange id - use to get list of valid exchanges ids

optional filters of requested historical data feed - check for each exchange and to get allowed channels and symbols for requested exchange

replay period start date (UTC) in a format, e.g., 2019-04-01

replay period end date (UTC) in a format, e.g., 2019-04-02

Streamed HTTP response provides data in NDJSON format (new line delimited JSON) - each response line is a JSON with market data message in plus local timestamp:

Exchanges' WebSocket APIs are designed to publish real-time market data feeds, not historical ones. Tardis-machine API fills that gap and allows "replaying" historical market data from any given past point in time with the same data format and 'subscribe' logic as real-time exchanges' APIs. In many cases existing exchanges' WebSocket clients can be used to connect to this endpoint just by changing URL, and receive historical market data in format for date ranges specified in URL query string params.

If two clients connect at the same time requesting data for different exchanges and provide the same , then data being send to those clients will be synchronized (by local timestamp).

In our preliminary benchmarks on AMD Ryzen 7 3700X, 64GB RAM, API endpoint was sending ~500 000 messages/s (already locally cached data).

You can also use existing WebSocket client, just by changing URL endpoint as shown in the example below that uses

As long as you already use existing WebSocket client that connects to and consumes real-time exchange market data feed, in most cases you can use it to connect to API as well just by changing URL endpoint.

requested exchange id - use to get list of valid exchanges ids

replay period start date (UTC) in a format, e.g., 2019-04-01

replay period end date (UTC) in a format, e.g., 2019-04-02

Normalized market data API endpoints provide data in across all supported exchanges. Both and APIs accept the same replay options payload via query string param. It's mostly matter of preference when choosing which protocol to use, but has also it's real-time counterpart , which connects directly to exchanges' real-time WebSocket APIs. This opens the possibility of seamless switching between real-time streaming and historical normalized market data replay.

Returns historical market data for specified via query string. Single streaming HTTP response returns data for the whole requested time period as . See which include normalized , , etc.

In our preliminary benchmarks on AMD Ryzen 7 3700X, 64GB RAM, API endpoint was returning ~100 000 messages/s and ~50 000 messages/s when order book snapshots were also requested.

See also official Tardis.dev library.

We're working on providing more samples and dedicated client libraries in different languages, but in the meanwhile to consume API responses in your language of choice, you should:

Provide url encoded JSON via options query string param when sending HTTP request

Parse each response line as JSON containing .

endpoint accepts required options query string param in url encoded JSON format.

requested exchange id - use to get list of valid exchanges ids

optional symbols of requested historical data feed - use to get allowed symbols for requested exchange

replay period start date (UTC) in a format, e.g., 2019-04-01

replay period end date (UTC) in a format, e.g., 2019-04-02

array of normalized for which historical data will be returned

when set to true, response includes messages that mark events when real-time WebSocket connection that was used to collect the historical data got disconnected

See .

Sends historical market data for specified via query string. See which include normalized , , etc.

is the real-time counterpart of this API endpoint, providing real-time market data in the same format, but not requiring API key as connects directly to exchanges' real-time WebSocket APIs.

See also official Tardis.dev library.

We're working on providing more samples and dedicated client libraries in different languages, but in the meanwhile to consume API responses in your language of choice, you should:

Provide url encoded JSON via options query string param when connecting to

endpoint

Parse each received WebSocket message as JSON containing .

endpoint accepts required options query string param in url encoded JSON format.

requested exchange id - use to get list of valid exchanges ids

optional symbols of requested historical data feed - use to get allowed symbols for requested exchange

replay period start date (UTC) in a format, e.g., 2019-04-01

replay period end date (UTC) in a format, e.g., 2019-04-02

array of normalized for which historical data will be provided

when set to true, sends also messages that mark events when real-time WebSocket connection that was used to collect the historical data got disconnected

In our preliminary benchmarks on AMD Ryzen 7 3700X, 64GB RAM, API endpoint was returning ~70 000 messages/s and ~40 000 messages/s when order book snapshots were also requested.

See .

Sends real-time market data for specified via query string. See which include normalized , , etc.

Provides consolidated real-time market data streaming functionality with as an array - provides single consolidated real-time data stream for all exchanges specified in .

is the historical counterpart of this API endpoint, providing historical market data in the same format.

See also official Tardis.dev library.

We're working on providing more samples and dedicated client libraries in different languages, but in the meanwhile to consume API responses in your language of choice, you should:

Provide url encoded JSON via options query string param when connecting to

endpoint

Parse each received WebSocket message as JSON containing .

endpoint accepts required options query string param in url encoded JSON format.

requested exchange id - use to get list of valid exchanges ids

array of normalized for which real-time data will be provided

when set to true, sends messages anytime underlying exchange real-time WebSocket connection(s) gets disconnected

See .

Initial order book snapshot (isSnapshot=true) plus incremental updates for each order book change. Please note that amount is the updated amount at that price level, not a delta. An amount of 0 indicates the price level can be removed.

Order book snapshot for selected number_of_levels (top bids and asks), snapshot_interval and . When snapshot_interval is set to 0 , snapshots are taken anytime order book state within specified levels has changed, otherwise snapshots are taken anytime snapshot_interval time has passed and there was an order book state change within specified levels. Order book snapshots are computed from exchanges' real-time order book streaming .

HTTP API
BitMEX
Deribit
Binance
Binance Futures
FTX
OKEx
Huobi Global
Huobi DM
bitFlyer
Bitstamp
Coinbase Pro
Kraken Futures
Gemini
Kraken
Bitfinex
Bybit
OKCoin
CoinFLEX
Open Source
tardisdev/tardis-machine image
bind mount
Python client
Node.js client
ccxws.
Node.js client
Node.js client
Node.js client
exchange-native market data APIs
HTTP
WebSocket
WebSocket API
normalized market data APIs
HTTP
WebSocket
historical market data replay across multiple exchanges
consolidated real-time data streaming
order book snapshots
trade bars
switching between real-time data streaming and historical data replay
environment variables
HTTP API
request options
WebSocket API
HTTP /replay
HTTP /replay
options object
exchange-native format
HTTP /replay
session key via query string param
WebSocket /ws-replay
/ws-replay
unified format
HTTP /replay-normalized
WebSocket /ws-replay-normalized
WebSocket /ws-replay-normalized API
/ws-stream-normalized
NDJSON
data types
supported data types
trade
order book change
customizable order book snapshots
HTTP /replay-normalized
HTTP /replay-normalized
options
normalized data messages
HTTP /replay-normalized
normalized data types
normalized
data types
supported data types
trade
order book change
customizable order book snapshots
WebSocket /ws-stream-normalized
WebSocket /ws-replay-normalized
options
WebSocket /ws-replay-normalized
normalized data
WebSocket /ws-replay-normalized
WebSocket /ws-replay-normalized
normalized data types
normalized
data types
supported data types
trade
order book change
customizable order book snapshots
options
options array
WebSocket /ws-replay-normalized
WebSocket /ws-stream-normalized
options
WebSocket /ws-stream-normalized
normalized data
WebSocket /ws-stream-normalized
normalized data types
trade
book_change
derivative_ticker
quote_{snapshot_interval}{time_unit}
book_snapshot_{number_of_levels}_{snapshot_interval}{time_unit}
trade_bar_{aggregation_interval}{suffix}
disconnect
http://localhost:8000/replay?options={%22exchange%22:%22bitmex%22,%22filters%22:[{%22channel%22:%22orderBookL2%22,%22symbols%22:[%22XBTUSD%22,%22ETHUSD%22]}],%22from%22:%222019-07-01%22,%22to%22:%222019-07-02%22}localhost
Click to see API response in the browser as long as tardis-machine is running on localhost:8000
http://localhost:8000/replay-normalized?options={%22exchange%22:%22bitmex%22,%22from%22:%222019-10-01%22,%22to%22:%222019-10-02%22,%22symbols%22:[%22XBTUSD%22,%22ETHUSD%22],%22withDisconnectMessages%22:true,%22dataTypes%22:[%22trade%22,%22book_change%22,%22book_snapshot_10_100ms%22]}localhost
Click to see API response in the browser as long as tardis-machine is running on localhost:8000
Tardis.dev
cluster of Node.js processes
Tardis.dev
cluster of Node.js processes
ISO 8601
ISO 8601
ISO 8601
ISO 8601
ISO 8601
ISO 8601
ISO 8601
ISO 8601
data types
disconnect
data types
disconnect
data types
disconnect
Tardis-machine
HTTP API
npm
Docker
NDJSON
WebSocket /ws-replay
time_unit
GitHub - tardis-dev/tardis-machine: Locally runnable server with built-in data caching, providing both tick-level historical and consolidated real-time cryptocurrency market data via HTTP and WebSocket APIsGitHub
Tardis-machine GitHub repository
historical data details
exchange-native format
exchange-native format
exchange-native format
exchange-native format
exchange-native
L2 (market by price)
L2 data (market by price)
/exchanges HTTP API
/exchanges/:exchange HTTP API
/exchanges HTTP API
/exchanges HTTP API
/exchanges/:exchange HTTP API
/exchanges HTTP API
/exchanges/:exchange HTTP API
/exchanges HTTP API
Logo