Node.js Client

Convenient access to tick-level historical and real-time cryptocurrency market data via Node.js

Introduction
Node.js tardis-dev library provides convenient access to tick-level historical and real-time cryptocurrency market data both in exchange-native and normalized formats. Instead of callbacks it relies on async iteration (for await ...of) enabling composability features like seamless switching between real-time data streaming and historical data replay or computing derived data locally.
​
historical market data replay
real-time market data streaming
const tardis = require('tardis-dev')
const { replayNormalized, normalizeTrades, normalizeBookChanges } = tardis
​
const messages = replayNormalized(
  {
    exchange: 'bitmex',
    symbols: ['XBTUSD', 'ETHUSD'],
    from: '2019-05-01',
    to: '2019-05-02'
  },
  normalizeTrades,
  normalizeBookChanges
)
​
for await (const message of messages) {
  console.log(message)
}
const tardis = require('tardis-dev')
const { streamNormalized, normalizeTrades, normalizeBookChanges } = tardis
​
const messages = streamNormalized(
  {
    exchange: 'bitmex',
    symbols: ['XBTUSD', 'ETHUSD']
  },
  normalizeTrades,
  normalizeBookChanges
)
​
for await (const message of messages) {
  console.log(message)
}
​​
​​
​
Features
​historical tick-level market data replay backed by Tardis.dev HTTP API — includes full order book depth snapshots plus incremental updates, trades, historical open interest, funding, index, mark prices, liquidations and more

consolidated real-time data streaming API connecting directly to exchanges' public WebSocket APIs

support for both exchange-native and normalized market data formats (unified format for accessing market data across all supported exchanges — normalized trades, order book and ticker data)

​seamless switching between real-time streaming and historical market data replay thanks to async iterables providing unified way of consuming data messages

transparent historical local data caching (cached data is stored on disk in compressed GZIP format and decompressed on demand when reading the data)

support for top cryptocurrency exchanges: BitMEX, Deribit, Binance, Binance Futures, FTX, OKEx, Huobi Global, Huobi DM, bitFlyer, Bitstamp, Coinbase Pro, Kraken Futures, Gemini, Kraken, Bitfinex, Bybit, OKCoin, CoinFLEX and more

automatic closed connections and stale connections reconnection logic for real-time streams

​combining multiple exchanges feeds into single one via combine helper function — synchronized historical market data replay and consolidated real-time data streaming from multiple exchanges

​computing derived data locally like order book imbalance, customizable trade bars, book snapshots and more via compute helper function and  computables, e.g., volume based bars, top 20 levels order book snapshots taken every 10 ms etc

​full limit order book reconstruction both for real-time and historical data via OrderBook object

fast and lightweight architecture —  low memory footprint and no heavy in-memory buffering

​extensible mapping logic that allows adjusting normalized formats for specific needs

built-in TypeScript support

​Open Source​
GitHub - tardis-dev/tardis-node: Convenient access to tick-level real-time and historical cryptocurrency market data via Node.js
GitHub
Tardis-dev GitHub repository
​
Examples (TOC)
​Streaming real-time spread across multiple exchanges

​Tick-by-tick replaying historical market data in exchange-native data format

​Tick-by-tick replaying historical market data in normalized data format

​Replaying large historical trades across multiple exchanges

​Computing customizable order book snapshots

​Seamless switching between real-time streaming and historical market data replay

​Real-time funding rate and open interest across multiple exchanges

​Saving historical funding, index and open interest data to CSV file

​Computing simple moving average of volume based trade bars​
​
​
​
Installation
Requires Node.js v12+ installed.
npm install tardis-dev --save
​
​
​
Debugging and logging
tardis-dev lib uses debug package for verbose logging and debugging purposes that can be enabled via DEBUG environment variable set to tardis-dev*.
​
​
​
Usage with TypeScript
Simply change from require
const { replay, stream } = require('tardis-dev')
to ES Modules import
import { replay, stream } from 'tardis-dev'
to enjoy first class TypeScript typings.
​
​
​
Replaying historical market data
See historical data details page to get detailed information about historical market data available for each exchange. 
​
replay(options)
Replays historical market data messages for given replay options in exchange-native format. Historical market data is being fetched efficiently (in parallel) from the Tardis.dev HTTP API and cached locally.  Returns async iterable.
const { replay } = require('tardis-dev')
​
const messages = replay({
  exchange: 'bitmex',
  filters: [
    { channel: 'trade', symbols: ['XBTUSD'] },
    { channel: 'orderBookL2', symbols: ['XBTUSD'] }
  ],
  from: '2019-05-01',
  to: '2019-05-02'
})
​
for await (const message of messages) {
  console.log(message)
}
​​
​​
​
​stream(options) is the real-time counterpart of replayfunction, returning real-time market data in the same format.
​
replay options
name
type
default 
​
exchange
string
-
requested exchange id - one of allowed values​
filters
{channel:string, symbols?: string[]}[]
[]
optional filters of requested historical data feed - use getExchangeDetails function to get allowed channels and symbols ids for requested exchange
from
string
-
replay period start date (UTC)  in a format recognized by the Date.parse(), e.g., 2019-04-01
to
string
-
replay period end date (UTC)  in a format recognized by the Date.parse(), e.g., 2019-04-02
skipDecoding
boolean | undefined
undefined
when set to true returns messages as buffers instead of decoding them to objects
withDisconnects
boolean | undefined
undefined
when set to true returns message with value undefined for events when connection that was recording the historical data got disconnected
apiKey
string | undefined
undefined
API key for Tardis.dev HTTP API - if not provided only first day of each month of historical data is accessible. It can also be set via init function for all replay calls.
​
type of messages provided by replay iterator (for await ...of)
type Message =
  | {
      localTimestamp: Date // local timestamp when message has been received
      message: any // message in exchange-native data format
    }
    // when skipDecoding is set to true
  | {
      localTimestamp: Buffer
      message: Buffer
    }
​
// when withDisconnects is set to true whole message can be undefined (disconnect)
Message | undefined
​
sample message
{
  localTimestamp: 2019-05-01T00:00:07.013Z,
  message: {
    table: 'orderBookL2',
    action: 'update',
    data: [ { symbol: 'XBTUSD', id: 8799474100, side: 'Buy', size: 167340 } ]
  }
}
​
​
​
replayNormalized(options, ...normalizers)
Replays historical market data messages for given replay options and normalizes messages using  normalizers provided as rest arguments.  Historical market data is being fetched efficiently (in parallel) from the Tardis.dev HTTP API and cached locally.  Returns async iterable.
const tardis = require('tardis-dev')
const { replayNormalized, normalizeTrades, normalizeBookChanges } = tardis
​
const messages = replayNormalized(
  {
    exchange: 'bitmex',
    symbols: ['XBTUSD', 'ETHUSD'],
    from: '2019-05-01',
    to: '2019-05-02'
  },
  normalizeTrades,
  normalizeBookChanges
)
​
for await (const message of messages) {
  console.log(message)
}
​​
​​
​
​streamNormalized(options, ...normalizers) is the real-time counterpart of replayNormalized function, returning real-time market data in the same format.
​
replay normalized options
name
type
default 
​
exchange
string
-
requested exchange id - one of allowed values​
symbols
string[] | undefined
undefined
optional symbols for requested  data feed - use getExchangeDetails function to get allowed symbols ids for requested exchange
from
string
-
replay period start date  (UTC) in a format recognized by the Date.parse(), e.g., 2019-04-01
to
string
-
replay period end  date  (UTC)  in a format recognized by the Date.parse() e.g., 2019-04-02
withDisconnectMessages
boolean | undefined
undefined
when set to true returns disconnect messages for events when connection that was recording the historical data got disconnected
apiKey
string | undefined
undefined
API key for Tardis.dev HTTP API - if not provided only first day of each month of historical data is accessible. It can also be set via init function for all replayNormalized calls.
​
Built-in normalizers
replayNormalized function accepts any number of normalizers as rest parameters that map from exchange-native format to normalized data format. tardis-dev ships with built in ones that normalize trades, order book and derivative ticker data but also allows adding custom ones.
​
types of messages provided by replayNormalized iterator (for await ...of)
Message types and formats depend on specific normalizers provided to replayNormalized function and are documented in detail in data normalization section.
​
sample message
Sample message produced by normalizeBookChanges 
{
  type: 'book_change',
  symbol: 'XBTUSD',
  exchange: 'bitmex',
  isSnapshot: false,
  bids: [{ price: 5263.5, amount: 1780043 }],
  asks: [],
  timestamp: 2019-05-01T00:00:04.430Z,
  localTimestamp: 2019-05-01T00:00:04.430Z
}
​
​
​
Streaming real-time market data
stream(options)
Streams real-time market data messages for given stream options in exchange-native format. It connects directly to exchanges WebSocket APIs and transparently restarts closed, broken or stale connections (open connections without data being send for specified amount of time).  Returns async iterable.
const { stream } = require('tardis-dev')
​
const messages = stream({
  exchange: 'bitmex',
  filters: [
    { channel: 'trade', symbols: ['XBTUSD'] },
    { channel: 'orderBookL2', symbols: ['XBTUSD'] }
  ]
})
​
for await (const message of messages) {
  console.log(message)
}
​​
​​
​
​replay(options) is the historical market data counterpart of stream function, returning historical market data in the same format.
​
stream options
name
type
default 
​
exchange
string
-
requested exchange id - one of  allowed values​
filters
{channel:string, symbols?: string[]}[]
[]
optional filters of requested real-time data feed - use getExchangeDetails to get allowed channels and symbols ids for requested exchange
skipDecoding
boolean | undefined
undefined
when set to true returns messages as buffers instead of decoding them to objects
withDisconnects
boolean | undefined
undefined
when set to true returns message with value undefined for real-time stream disconnect events
timeoutIntervalMS
number
10000
specifies time in milliseconds after which connection is restarted if no message has been received from the exchange
onError
(err) => void | undefined
undefined
Optional callback invoked when real-time WebSocket  connection error occurs, useful for custom error logging etc.
​
type of messages provided by stream iterator (for await ...of)
type Message =
  | {
      localTimestamp: Date // local timestamp when message has been received
      message: any // message in exchange-native data format
    }
    // when skipDecoding is set to true
  | {
      localTimestamp: Buffer
      message: Buffer
    }
​
// when withDisconnects is set to true whole message can be undefined
Message | undefined
​
sample message
{
  localTimestamp: 2019-10-22T09:24:51.025Z,
  message: {
    table: 'orderBookL2',
    action: 'update',
    data: [
      { symbol: 'XBTUSD', id: 8799172400, side: 'Sell', size: 369501 }
    ]
  }
}
​
​
​
streamNormalized(options, ...normalizers)
Streams real-time market data messages for given stream options and normalizes messages using provided normalizers provided as rest arguments. It connects directly to exchanges WebSocket APIs and transparently restarts closed, broken or stale connections (open connections without data being send for specified amount of time).  Returns async iterable.
const tardis = require('tardis-dev')
const { streamNormalized, normalizeTrades, normalizeBookChanges } = tardis
​
const messages = streamNormalized(
  {
    exchange: 'bitmex',
    symbols: ['XBTUSD', 'ETHUSD']
  },
  normalizeTrades,
  normalizeBookChanges
)
​
for await (const message of messages) {
  console.log(message)
}
​​
​​
​
​replayNormalized(options) is the historical counterpart of streamNormalized function, returning historical market data in the same format.
​
stream normalized options
name
type
default 
​
exchange
string
-
requested exchange id - one of allowed values​
symbols
string[] | undefined
undefined
instruments symbols for requested  data feed
withDisconnectMessages
boolean | undefined
undefined
when set to true returns disconnect messages for real-time stream disconnect events
timeoutIntervalMS
number
10000
specifies time in milliseconds after which connection is restarted if no message has been received from the exchange
onError
((err) => void) | undefined
undefined
Optional callback invoked when real-time WebSocket  connection or mapping error occurs, useful for custom error logging etc.
​
Built-in normalizers
streamNormalized function can  accept any number of custom normalizers as rest parameters that map from exchange-native format to normalized data format. tardis-dev ships with built in ones that normalize trades, order book and derivative ticker data but also allows adding custom ones.
​
types of messages provided by streamNormalized iterator (for await ...of)
Message types and formats depend on specific normalizers provided to streamNormalized function and are documented in detail in data normalization section.
​
sample message
Sample message produced by normalizeTrades 
{
  type: 'trade',
  symbol: 'XBTUSD',
  exchange: 'bitmex',
  id: 'b1f4b309-80e2-1ffb-340b-2f7576f6ef0d',
  price: 7447.5,
  amount: 100,
  side: 'buy',
  timestamp: 2019-10-24T13:28:07.867Z,
  localTimestamp: 2019-10-24T13:28:07.938Z
}
​
​
​
Historical market data helpers
init(options)
This function doesn't affect real-time streaming functionality in any way, it's useful only for historical data replay.
When working with market data viareplay and replayNormalized functions by default only first day of each month of historical data is available for replay  as well as locally cached historical data is stored in default location on disk (OS temp dir). 
Init function allows providing apiKey received via email after ordering historical market data access via Tardis.dev website as well as customcacheDir. ApiKey can also be provided directly via options of replay and replayNormalized functions - that overrides anything that was provided via init.
const { init } = require('tardis-dev')
​
init({
  apiKey: 'YOUR API KEY',
  cacheDir: 'CUSTOM CACHE DIR PATH'
})
​
init options
name
type
defaults
​
apiKey
string | undefined
undefined
API key for Tardis.dev HTTP API - if not provided only first day of each month of historical data is accessible
cacheDir
string
<os.tmpdir>/.tardis-cache
path to local dir that will be used as cache location - if not provided default temp dir for given OS will be used
​
​
​
getExchangeDetails(exchange)
Given exchange id provides exchange details (available symbols, availability dates, available channels, pricing info etc) provided by exchanges/:exchange API endpoint.
const { getExchangeDetails } = require('tardis-dev')
​
const bitmexExchangeDetails = await getExchangeDetails('bitmex')
​​
​​
​
type of response returned by awaiting on getExchangeDetails
{
  id: string
  name: string
  enabled: boolean
  filterable: boolean
  availableSince: string
  availableSymbols: {
    id: string
    type: 'spot' | 'future' | 'perpetual' | 'option'
    availableSince: string
    availableTo?: string
  }[]
  availableChannels: string[]
  incidentReports: {
    from: string
    to: string
    status: string
    details: string
  }
}
​
​
​
getApiKeyAccessInfo(apiKey?)
Given apiKey provided as optional parameter or provided in init function provides information about what historical data is available for it - exchanges, date ranges, symbols.
const { getApiKeyAccessInfo } = require('tardis-dev')
​
const details = await getApiKeyAccessInfo('YOUR_API_KEY')
​
type of response returned by awaiting on getApiKeyAccessInfo()
{
  exchange: string
  from: string
  to: string
  symbols: string[]
}[]
​
​
​
clearCache()
Clears local data cache dir.
const { clearCache } = require('tardis-dev')
​
await clearCache()
​
​
​
Data normalization
Data normalization allows consuming market data feeds from various exchanges in consistent format.
tardis-dev has following built-in normalizers that can be provided to replayNormalized or streamNormalized functions:
​normalizeTrades - provides normalized trade data
 normalizeBookChanges - provides normalized book_change data
​normalizeDerivativeTickers - provides normalized funding, index and mark price data
If you're interested in how exactly data is mapped from exchange-native format to normalized one, please follow code in tardis-dev GitHub repository for each exchange and if you determined that mapping should be done differently please read "modifying built-in and adding custom normalizers" section.
const { streamNormalized, normalizeTrades, normalizeBookChanges,
  normalizeDerivativeTickers } = require('tardis-dev')
​
// or replayNormalized to replay normalized historical data
const messages = streamNormalized(
  {
    exchange: 'deribit',
    symbols: ['BTC-PERPETUAL']
  },
  normalizeTrades,
  normalizeBookChanges,
  normalizeDerivativeTickers
)
​
for await (const message of messages) {
  if (message.type === 'book_change') {
    // process normalized book change
  }
  if (message.type === 'trade') {
    // process normalized trade
  }
  if (message.type === 'derivative_ticker') {
    // process normalized derivative_ticker
  }
}
​​
​​
​
​
normalizeTrades
When passed as an arg toreplayNormalized or streamNormalized function provides normalized trade data for all supported exchanges. 
sample message
type definition
{
  type: 'trade',
  symbol: 'XBTUSD',
  exchange: 'bitmex',
  id: '282a0445-0e3a-abeb-f403-11003204ea1b',
  price: 7996,
  amount: 50,
  side: 'sell',
  timestamp: 2019-10-23T10:32:49.669Z,
  localTimestamp: 2019-10-23T10:32:49.740Z
}
{
  type: 'trade'
  symbol: string
  exchange: string
  id: string | undefined
  price: number
  amount: number
  side: 'buy' | 'sell' | 'unknown' // liquidity taker side (aggressor)
  timestamp: Date
  localTimestamp: Date
}
​
​
normalizeBookChanges
When passed as an arg toreplayNormalized or streamNormalized function provides normalized book_change data for all supported exchanges. 
Provides initial L2 (market by price) order book snapshots (isSnapshot=true) plus incremental updates for each order book change.  Please note that amount is the updated amount at that price level, not a delta. An amount of 0 indicates the price level can be removed.
sample message
type definition
{
  type: 'book_change',
  symbol: 'XBTUSD',
  exchange: 'bitmex',
  isSnapshot: false,
  bids: [],
  asks: [{ price: 7985, amount: 283318 }],
  timestamp: 2019-10-23T11:29:53.469Z,
  localTimestamp: 2019-10-23T11:29:53.469Z
}
{
  type: 'book_change'
  symbol: string
  exchange: string
  isSnapshot: boolean
  bids: { price: number; amount: number }[]
  asks: { price: number; amount: number }[]
  timestamp: Date
  localTimestamp: Date
}
​
​
normalizeDerivativeTickers
When passed as an arg toreplayNormalized or streamNormalized function provides normalized derivative_ticker data for supported exchanges that trade derivative instruments. 
sample message
type definition
{
  type: 'derivative_ticker',
  symbol: 'BTC-PERPETUAL',
  exchange: 'deribit',
  lastPrice: 7987.5,
  openInterest: 84129491,
  fundingRate: -0.00001568,
  indexPrice: 7989.28,
  markPrice: 7987.56,
  timestamp: 2019-10-23T11:34:29.302Z,
  localTimestamp: 2019-10-23T11:34:29.416Z
}

Node.js Client

Introduction

Features

Examples (TOC)

Installation

Debugging and logging

Usage with TypeScript

Replaying historical market data

`replay(options)`

replay options

type of messages provided by `replay` iterator (`for await ...of`)

sample message

`replayNormalized(options, ...normalizers)`

replay normalized options

Built-in normalizers

types of messages provided by `replayNormalized` iterator (`for await ...of`)

sample message

Streaming real-time market data

`stream(options)`

stream options

type of messages provided by `stream` iterator (`for await ...of`)

sample message

`streamNormalized(options, ...normalizers)`

stream normalized options

Built-in normalizers

types of messages provided by `streamNormalized` iterator (`for await ...of`)

sample message

Historical market data helpers

`init(options)`

init options

`getExchangeDetails(exchange)`

type of response returned by awaiting on `getExchangeDetails`

`getApiKeyAccessInfo(apiKey?)`

type of response returned by awaiting on `getApiKeyAccessInfo()`

`clearCache()`

Data normalization

`normalizeTrades`

`normalizeBookChanges`

`normalizeDerivativeTickers`

name	type	default
exchange	string	-	requested exchange id - one of allowed values
filters	{channel:string, symbols?: string[]}[]	[]	optional filters of requested historical data feed - use `getExchangeDetails` function to get allowed channels and symbols ids for requested exchange
from	string	-	replay period start date (UTC) in a format recognized by the Date.parse(), e.g., `2019-04-01`
to	string	-	replay period end date (UTC) in a format recognized by the Date.parse(), e.g., `2019-04-02`
skipDecoding	boolean \| undefined	undefined	when set to `true` returns messages as buffers instead of decoding them to objects
withDisconnects	boolean \| undefined	undefined	when set to `true` returns message with value `undefined` for events when connection that was recording the historical data got disconnected
apiKey	string \| undefined	undefined	API key for Tardis.dev HTTP API - if not provided only first day of each month of historical data is accessible. It can also be set via `init` function for all replay calls.

name	type	defaults
apiKey	string \| undefined	undefined	API key for Tardis.dev HTTP API - if not provided only first day of each month of historical data is accessible
cacheDir	string	<os.tmpdir>/.tardis-cache	path to local dir that will be used as cache location - if not provided default `temp` dir for given OS will be used