Tardis.dev Documentation

Getting Started

CSV Data Export

Order Form

Support

Tardis-Machine

TODO: description

This is in progress draft of tardis-machine v2 documentation. Please contact us in case of any questions. We're fully aware documentation is not yet complete, working hard to finish it as soon as possible.
Introduction
​Tardis-machine is a fast, locally runnable server with built-in local caching, that uses lower level HTTP API under the hood. It can be installed via NPM or run as a Docker Container  (image available on Docker hub).
Locally cached data is stored on disk in compressed form (GZIP) and decompressed on demand when reading the data.
Tardis-machine provides on-demand tick-level market data replay API from any moment in time in exchange's WebSocket data format both via streaming HTTP and WebSocket endpoints. In many cases existing exchanges WebSocket clients can be used to consume historical message stream as if it was real-time one.
​
TODO : list of features
// transparent open source real-time data , hosted locally
​
​
See historical data details page to get detailed information about historical market data available for each exchange. 
Installation
 via Docker
Run command below in order to pull and run latest version of tardisdev/tardis-machine image.  Tardis-machine server will be available on host via 8000 port. Your  API key will be passed via env variable (TM_API_KEY) — simply replace <YOUR_API_KEY> with API key you've received.
### running without persitent local cache
docker run -p 8000:8000 -e "TM_API_KEY=<YOUR_API_KEY>" -d tardisdev/tardis-machine
Command above does not use persistent volumes for local caching (each docker restart will result in loosing local cache). In order to use for example./host-cache-dir as persistent volume (bind mount) cache directory, run: 
docker run -v ./host-cache-dir:/.cache -p 8000:8000 -e "TM_API_KEY=<YOUR_API_KEY>" -d tardisdev/tardis-machine
Since using volumes can cause issues especially on Windows, it's perfectly fine to run docker without them, with the caveat of potentially poor local cache ratio after container restart.
​
​
​
via npx
Requires Node.js v12+ installed. 
Run command below to install and start tardis-machine server running on port 8000 by default (port can be changed --port CLI flag)
npx tardis-machine --api-key=<YOUR_API_KEY>
If you'd like to try server on sample data (first full day of each month) run the command without --api-key option.
npx tardis-machine
Run npx tardis-machine --help to see all available CLI options (setting custom cache directory, enabling debug logs etc.)
​
​
​
via npm 
Requires Node.js v12+ installed. 
npm install -g tardis-machine
Run command below to start tardis-machine server running on port 8000 by default (port can be changed --port CLI flag)
tardis-machine --api-key=<YOUR_API_KEY>
If you'd like to try server on sample data (first full day of each month) run the command without --api-key option.
tardis-machine
Run tardis-machine --help to see all available CLI options (setting custom cache directory, enabling debug logs etc.)
​
​
​
Debugging and logging
In order to see detailed logs & debug messages when using Docker, run:
docker run -v ./host-cache-dir:/.cache -p 8000:8000 -e "TM_API_KEY=YOUR_API_KEY" -e "DEBUG=tardis-dev:machine*" -d tardisdev/tardis-machine
Setting DEBUG=tardis-dev:machine* env variable will cause to printing debug logs to stdout.
​
When running via npm installation or npx  you can pass --debug flag when starting CLI or set DEBUG environment variable to tardis-dev:machine*.
npx tardis-machine --debug
# or when installed globally via npm
tardis-machine --debug
​
​
​
HTTP endpoints
Base HTTP API Endpoint: http://localhost:8000 (assuming default options when starting tardis-machine)
​
GET /replay?options={options}
Replays historical market data messages for given replay options query param in exchange native format. Historical raw market data is being fetched efficiently (in parallel) from the HTTP API and cached locally by tardis-machine.
Accepts required options query string param in url encoded JSON format.
Returns messages in NDJSON format (new line delimited JSON) - each line is JSON with message in exchange native format.
​
replay options
name
type
default 
​
exchange
string
-
requested exchange id - one of allowed values​
filters
{channel:string, symbols?: string[]}[]
[]
optional filters of requested historical data feed - use getExchangeDetails function to get allowed channels and symbols ids for requested exchange
from
string
-
replay period start date (UTC)  in a format recognized by the Date.parse(), e.g., 2019-04-01
to
string
-
replay period end date (UTC)  in a format recognized by the Date.parse(), e.g., 2019-04-02
skipDecoding
boolean | undefined
undefined
when set to true returns messages as buffers instead of decoding them to objects
withDisconnects
boolean | undefined
undefined
when set to true returns message with value undefined for events when connection that was recording the historical data was closed
apiKey
string | undefined
undefined
 API key for tardis.dev API - if not provided only first day of each month of historical data is accessible. It can also be set via init function for all replay calls.
​
​
TODO: response format
TODO: example response
​
​
​
​
GET /replay-normalized?options={options}
​
​
​
Replay normalized options JSON string needs to be url encoded when provided via query string
​
Replay normalized options JSON
{
  exchange: string
  symbols?: string[] | undefined
  from: string
  to: string
  withDisconnectMessages?: boolean | undefined
  apiKey?: boolean | undefined
  dataTypes: string[]
}
// or array of options 
​
Example request
const serialize = options => {
  return encodeURIComponent(JSON.stringify(options))
}
​
const options = {
  exchange: 'bitmex',
  symbols: ['ETHUSD'],
  from: '2019-06-01',
  to: '2019-06-02',
  dataTypes: ['trade', 'book_change', 'derivative_ticker']
}
​
​
fetch(`http://localhost:8000/replay-normalized?options=${serialize(options)}`)
​
​
​
​
​
WebSocket Endpoints
Base WebSocket API Endpoint: ws://localhost:8000 (assuming default options when starting tardis-machine)
​
WS /ws-replay
...
WS /ws-replay-normalized
...
WS /ws-stream-normalized
...
Normalized data types
Accepted data types strings
trade
book_change
derivative_ticker
quote
quote_{suffix}
book_snapshot_{numberof_top_book_levels}_{snapshot_interval}{suffix}
Available suffixes:
ms
s
m
trade_bar_{aggregation_interval}{suffix}
Available suffixes:
ms
s
m
ticks
vol
​
​
trade
{
  type: 'trade'
  symbol: string
  exchange: string
  id: string | undefined
  price: number
  amount: number
  side: 'buy' | 'sell' | 'unknown' // liquidity taker side (aggressor)
  timestamp: string
  localTimestamp: string
}
​
sample normalized 
trade JSON message
{
  "type": "trade",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "id": "282a0445-0e3a-abeb-f403-11003204ea1b",
  "price": 7996,
  "amount": 50,
  "side": "sell",
  "timestamp": "2019-10-23T10:32:49.669Z",
  "localTimestamp": "2019-10-23T10:32:49.740Z"
}
​
​
​
book_change
{
  type: 'book_change'
  symbol: string
  exchange: string
  isSnapshot: boolean
  bids: { price: number; amount: number }[]
  asks: { price: number; amount: number }[]
  timestamp: string
  localTimestamp: string
}
​
sample normalized book_change JSON message
{
  "type": "book_change",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "isSnapshot": false,
  "bids": [],
  "asks": [
    {
      "price": 7985,
      "amount": 283318
    }
  ],
  "timestamp": "2019-10-23T11:29:53.469Z",
  "localTimestamp": "2019-10-23T11:29:53.469Z"
}
​
​
​
derivative_ticker
{
  type: 'derivative_ticker'
  symbol: string
  exchange: string
  lastPrice: number | undefined
  openInterest: number | undefined
  fundingRate: number | undefined
  indexPrice: number | undefined
  markPrice: number | undefined
  timestamp: string
  localTimestamp: string
}
​
sample normalized derivative_ticker JSON message
{
  "type": "derivative_ticker",
  "symbol": "BTC-PERPETUAL",
  "exchange": "deribit",
  "lastPrice": 7987.5,
  "openInterest": 84129491,
  "fundingRate": -0.00001568,
  "indexPrice": 7989.28,
  "markPrice": 7987.56,
  "timestamp": "2019-10-23T11:34:29.302Z",
  "localTimestamp": "2019-10-23T11:34:29.416Z"
}
​
​
​
trade_bar
{
  type: 'trade_bar'
  symbol: string
  exchange: string
  name: string
  interval: number
  kind: 'time' | 'volume' | 'tick'
  open: number
  high: number
  low: number
  close: number
  volume: number
  buyVolume: number
  sellVolume: number
  trades: number
  vwap: number
  openTimestamp: Date
  closeTimestamp: Date
  timestamp: Date
  localTimestamp: Date
}
​
sample normalized trade_bar JSON message
{
  "type": "trade_bar",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "name": "trade_bar_10000ms",
  "interval": 10000,
  "kind": "time",
  "open": 7623.5,
  "high": 7623.5,
  "low": 7623,
  "close": 7623.5,
  "volume": 37034,
  "buyVolume": 24244,
  "sellVolume": 12790,
  "trades": 9,
  "vwap": 7623.327320840309,
  "openTimestamp": "2019-10-25T13:11:31.574Z",
  "closeTimestamp": "2019-10-25T13:11:39.212Z",
  "localTimestamp": "2019-10-25T13:11:40.369Z",
  "timestamp": "2019-10-25T13:11:40.000Z"
}
​
​
​
book_snapshot
{
  type: 'book_snapshot'
  symbol: string
  exchange: string
  name: string
  depth: number
  interval: number
  bids: { price: number; amount: number }[]
  asks: { price: number; amount: number }[]
  timestamp: Date
  localTimestamp: Date
}
​
sample normalized book_snapshot JSON message
{
  "type": "book_snapshot",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "name": "book_snapshot_2_50ms",
  "depth": 2,
  "interval": 50,
  "bids": [
    {
      "price": 7633.5,
      "amount": 1906067
    },
    {
      "price": 7633,
      "amount": 65319
    }
  ],
  "asks": [
    {
      "price": 7634,
      "amount": 1467849
    },
    {
      "price": 7634.5,
      "amount": 67939
    }
  ],
  "timestamp": "2019-10-25T13:39:46.950Z",
  "localTimestamp": "2019-10-25T13:39:46.961Z"
}
​
​
​
Examples
​
​
Example below  illustrates tardis-machine executing inline from code and official Node.js BitMEX WebSocket client communicating with it and consuming historical data as if it was real-time stream - it can as well run as CLI installed via NPM and as Docker container.
tardis-machine example working with official Node.js BitMEX WebSocket client 
Try example above using run button  - first change Node version to v12 in dropdown on the left
​
tardis-machine example working together with ccxws lib
​
Try example above using run button  - first change Node version to v12 in dropdown on the left

API - Previous

Python

with Python

Last updated 5 days ago

Tardis-Machine

Introduction

Installation

via Docker

via npx

via npm

Debugging and logging

HTTP endpoints

`GET` /replay?options={options}

replay options

`GET` /replay-normalized?options={options}

Replay normalized options JSON

Example request

WebSocket Endpoints

WS /ws-replay

WS /ws-replay-normalized

WS /ws-stream-normalized

Normalized data types

Accepted data types strings

`trade`

sample normalized

`trade` JSON message

`book_change`

sample normalized `book_change` JSON message

`derivative_ticker`

sample normalized `derivative_ticker` JSON message

`trade_bar`

sample normalized `trade_bar` JSON message

`book_snapshot`

sample normalized `book_snapshot` JSON message

Examples

name	type	default
exchange	string	-	requested exchange id - one of allowed values
filters	{channel:string, symbols?: string[]}[]	[]	optional filters of requested historical data feed - use `getExchangeDetails` function to get allowed channels and symbols ids for requested exchange
from	string	-	replay period start date (UTC) in a format recognized by the Date.parse(), e.g., `2019-04-01`
to	string	-	replay period end date (UTC) in a format recognized by the Date.parse(), e.g., `2019-04-02`
skipDecoding	boolean \| undefined	undefined	when set to `true` returns messages as buffers instead of decoding them to objects
withDisconnects	boolean \| undefined	undefined	when set to `true` returns message with value `undefined` for events when connection that was recording the historical data was closed
apiKey	string \| undefined	undefined	API key for tardis.dev API - if not provided only first day of each month of historical data is accessible. It can also be set via `init` function for all replay calls.

Tardis-Machine

Introduction

Installation

via Docker

via npx

via npm

Debugging and logging

HTTP endpoints

GET /replay?options={options}

replay options

GET /replay-normalized?options={options}

​

Replay normalized options JSON

Example request

​

WebSocket Endpoints

WS /ws-replay

WS /ws-replay-normalized

WS /ws-stream-normalized

Normalized data types

Accepted data types strings

trade

sample normalized

trade JSON message

book_change

sample normalized book_change JSON message

derivative_ticker

sample normalized derivative_ticker JSON message

trade_bar

sample normalized trade_bar JSON message

book_snapshot

sample normalized book_snapshot JSON message

Examples

​

`GET` /replay?options={options}

`GET` /replay-normalized?options={options}

`trade`

`trade` JSON message

`book_change`

sample normalized `book_change` JSON message

`derivative_ticker`

sample normalized `derivative_ticker` JSON message

`trade_bar`

sample normalized `trade_bar` JSON message

`book_snapshot`

sample normalized `book_snapshot` JSON message