tardis.dev docs
tardis.dev docs
Getting Started
FAQ
Order
Support
Welcome
FAQ
API
Getting Started
Node.js
Python
Tardis-Machine
with Python
with R
with Rust
with .NET
with Julia
with Go
HTTP API
Historical Data Details
Legal
Privacy Policy
Terms of Service
External Links
GitHub
Support Live Chat
Contact
Tardis.dev
Powered by GitBook

Tardis-Machine

Introduction

​Tardis-machine is a fast, locally runnable server with built-in local caching, that uses lower level HTTP API under the hood. It can be installed via NPM or run as a Docker Container (image available on Docker hub).

Locally cached data is stored on disk in compressed form (GZIP) and decompressed on demand when reading the data.

Tardis-machine provides on-demand tick-level market data replay API from any moment in time in exchange's WebSocket data format both via streaming HTTP and WebSocket endpoints. In many cases existing exchanges WebSocket clients can be used to consume historical message stream as if it was real-time one.

​

​

​

Installation

● via Docker

Run command below in order to pull and run latest version of tardisdev/tardis-machine image. Tardis-machine server will be available on host via 8000 port. Your API key will be passed via env variable (TM_API_KEY) — simply replace <YOUR_API_KEY> with API key you've received.

### running without persitent local cache
docker run -p 8000:8000 -e "TM_API-KEY=<YOUR_API_KEY>" -d tardisdev/tardis-machine

Command above does not use persistent volumes for local caching (each docker restart will result in loosing local cache). In order to use for example./host-cache-dir as persistent volume (bind mount) cache directory, run: 

docker run -v ./host-cache-dir:/.cache -p 8000:8000 -e "TM_API-KEY=<YOUR_API_KEY>" -d tardisdev/tardis-machine

Since using volumes can cause issues especially on Windows, it's perfectly fine to run docker without them, with the caveat of potentially poor local cache ratio after container restart.

​

​

​

● via npx

Requires Node.js v12+ installed.

Run command below to install and start tardis-machine server running on port 8000 by default (port can be changed --port CLI flag)

npx tardis-machine --api-key=<YOUR_API_KEY>

If you'd like to try server on sample data (first full day of each month) run the command without --api-key option.

npx tardis-machine

Run npx tardis-machine --help to see all available CLI options (setting custom cache directory, enabling debug logs etc.)

​

​

​

● via npm

Requires Node.js v12+ installed. 

npm install -g tardis-machine

Run command below to start tardis-machine server running on port 8000 by default (port can be changed --port CLI flag)

tardis-machine --api-key=<YOUR_API_KEY>

If you'd like to try server on sample data (first full day of each month) run the command without --api-key option.

tardis-machine

Run tardis-machine --help to see all available CLI options (setting custom cache directory, enabling debug logs etc.)

​

​

​

Debugging and logging

In order to see detailed logs & debug messages when using Docker, run:

docker run -v ./host-cache-dir:/.cache -p 8000:8000 -e "TM_API-KEY=YOUR_API_KEY" -e="DEBUG=tardis-dev:machine*" -d tardisdev/tardis-machine

Setting DEBUG=tardis-dev:machine* env variable will cause to printing debug logs to stdout.

​

When running via npm installation or npx you can pass --debug flag when starting CLI or set DEBUG environment variable to tardis-dev:machine*.

npx tardis-machine --debug
# or when installed globally via npm
tardis-machine --debug

​

​

​

Available HTTP API endpoints

get
/replay

http://localhost:8000/replay?options={options}
Request
Response
Path Parameters
optional
string
​
200: OK
​

​

​

​

get
/replay-normalized

http://localhost:8000/replay-normalized?options={options}
Request
Response
Query Parameters
options
required
string
Replay normalized options as url encoded JSON string
200: OK
​

Replay normalized options JSON string needs to be url encoded when provided via query string

​

Replay normalized options JSON

{
  exchange: string
  symbols?: string[] | undefined
  from: string
  to: string
  withDisconnectMessages?: boolean | undefined
  apiKey?: boolean | undefined
  dataTypes: string[]
}
// or array of options

​

Example request

const serialize = options => {
  return encodeURIComponent(JSON.stringify(options))
}
​
const options = {
  exchange: 'bitmex',
  symbols: ['ETHUSD'],
  from: '2019-06-01',
  to: '2019-06-02',
  dataTypes: ['trade', 'book_change', 'derivative_ticker']
}
​
​
fetch(`http://localhost:8000/replay-normalized?options=${serialize(options)}`)

​

Accepted data types strings

  • trade

  • book_change

  • derivative_ticker

  • quote

  • quote_{suffix}

  • book_snapshot_{numberof_top_book_levels}_{snapshot_interval}{suffix}

    • Available suffixes:

      • ms

      • s

      • m

  • trade_bar_{aggregation_interval}{suffix}

    • Available suffixes:

      • ms

      • s

      • m

      • ticks

      • vol

​

​

​

Available WebSocket API endpoints

/ws-replay

...

/ws-replay-normalized

...

/ws-stream-normalized

...

Normalized JSON data format

trade

{
  type: 'trade'
  symbol: string
  exchange: string
  id: string | undefined
  price: number
  amount: number
  side: 'buy' | 'sell' | 'unknown' // liquidity taker side (aggressor)
  timestamp: string
  localTimestamp: string
}

​

sample normalized

trade JSON message

{
  "type": "trade",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "id": "282a0445-0e3a-abeb-f403-11003204ea1b",
  "price": 7996,
  "amount": 50,
  "side": "sell",
  "timestamp": "2019-10-23T10:32:49.669Z",
  "localTimestamp": "2019-10-23T10:32:49.740Z"
}

​

​

​

book_change

{
  type: 'book_change'
  symbol: string
  exchange: string
  isSnapshot: boolean
  bids: { price: number; amount: number }[]
  asks: { price: number; amount: number }[]
  timestamp: string
  localTimestamp: string
}

​

sample normalized book_change JSON message

{
  "type": "book_change",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "isSnapshot": false,
  "bids": [],
  "asks": [
    {
      "price": 7985,
      "amount": 283318
    }
  ],
  "timestamp": "2019-10-23T11:29:53.469Z",
  "localTimestamp": "2019-10-23T11:29:53.469Z"
}

​

​

​

derivative_ticker

{
  type: 'derivative_ticker'
  symbol: string
  exchange: string
  lastPrice: number | undefined
  openInterest: number | undefined
  fundingRate: number | undefined
  indexPrice: number | undefined
  markPrice: number | undefined
  timestamp: string
  localTimestamp: string
}

​

sample normalized derivative_ticker JSON message

{
  "type": "derivative_ticker",
  "symbol": "BTC-PERPETUAL",
  "exchange": "deribit",
  "lastPrice": 7987.5,
  "openInterest": 84129491,
  "fundingRate": -0.00001568,
  "indexPrice": 7989.28,
  "markPrice": 7987.56,
  "timestamp": "2019-10-23T11:34:29.302Z",
  "localTimestamp": "2019-10-23T11:34:29.416Z"
}

​

​

​

trade_bar

{
  type: 'trade_bar'
  symbol: string
  exchange: string
  name: string
  interval: number
  kind: 'time' | 'volume' | 'tick'
  open: number
  high: number
  low: number
  close: number
  volume: number
  buyVolume: number
  sellVolume: number
  trades: number
  vwap: number
  openTimestamp: Date
  closeTimestamp: Date
  timestamp: Date
  localTimestamp: Date
}

​

sample normalized trade_bar JSON message

{
  "type": "trade_bar",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "name": "trade_bar_10000ms",
  "interval": 10000,
  "kind": "time",
  "open": 7623.5,
  "high": 7623.5,
  "low": 7623,
  "close": 7623.5,
  "volume": 37034,
  "buyVolume": 24244,
  "sellVolume": 12790,
  "trades": 9,
  "vwap": 7623.327320840309,
  "openTimestamp": "2019-10-25T13:11:31.574Z",
  "closeTimestamp": "2019-10-25T13:11:39.212Z",
  "localTimestamp": "2019-10-25T13:11:40.369Z",
  "timestamp": "2019-10-25T13:11:40.000Z"
}

​

​

​

book_snapshot

{
  type: 'book_snapshot'
  symbol: string
  exchange: string
  name: string
  depth: number
  interval: number
  bids: { price: number; amount: number }[]
  asks: { price: number; amount: number }[]
  timestamp: Date
  localTimestamp: Date
}

​

sample normalized book_snapshot JSON message

{
  "type": "book_snapshot",
  "symbol": "XBTUSD",
  "exchange": "bitmex",
  "name": "book_snapshot_2_50ms",
  "depth": 2,
  "interval": 50,
  "bids": [
    {
      "price": 7633.5,
      "amount": 1906067
    },
    {
      "price": 7633,
      "amount": 65319
    }
  ],
  "asks": [
    {
      "price": 7634,
      "amount": 1467849
    },
    {
      "price": 7634.5,
      "amount": 67939
    }
  ],
  "timestamp": "2019-10-25T13:39:46.950Z",
  "localTimestamp": "2019-10-25T13:39:46.961Z"
}

​

​

​

Examples

​

​

Example below illustrates tardis-machine executing inline from code and official Node.js BitMEX WebSocket client communicating with it and consuming historical data as if it was real-time stream - it can as well run as CLI installed via NPM and as Docker container.

tardis-machine example working with official Node.js BitMEX WebSocket client

Try example above using run button - first change Node version to v12 in dropdown on the left

​

tardis-machine example working together with ccxws lib

​

Try example above using run button - first change Node version to v12 in dropdown on the left

API - Previous
Python
Next
with Python
Last updated 1 month ago
Contents
Introduction
Installation
● via Docker
● via npx
● via npm
Debugging and logging
Available HTTP API endpoints
get
/replay
get
/replay-normalized
Accepted data types strings
Available WebSocket API endpoints
/ws-replay
/ws-replay-normalized
/ws-stream-normalized
Normalized JSON data format
trade
book_change
derivative_ticker
trade_bar
book_snapshot
Examples