Node.js Client
Convenient access to tick-level historical and real-time cryptocurrency market data via Node.js
Node.js
tardis-dev
library provides convenient access to tick-level historical and real-time cryptocurrency market data both in exchange-native and normalized formats. Instead of callbacks it relies on async iteration (for await ...of) enabling composability features like seamless switching between real-time data streaming and historical data replay or computing derived data locally.
historical market data replay
real-time market data streaming
const tardis = require('tardis-dev')
const { replayNormalized, normalizeTrades, normalizeBookChanges } = tardis
const messages = replayNormalized(
{
exchange: 'bitmex',
symbols: ['XBTUSD', 'ETHUSD'],
from: '2019-05-01',
to: '2019-05-02'
},
normalizeTrades,
normalizeBookChanges
)
for await (const message of messages) {
console.log(message)
}
const tardis = require('tardis-dev')
const { streamNormalized, normalizeTrades, normalizeBookChanges } = tardis
const messages = streamNormalized(
{
exchange: 'bitmex',
symbols: ['XBTUSD', 'ETHUSD']
},
normalizeTrades,
normalizeBookChanges
)
for await (const message of messages) {
console.log(message)
}
- historical tick-level market data replay backed by Tardis.dev HTTP API — includes full order book depth snapshots plus incremental updates, trades, historical open interest, funding, index, mark prices, liquidations and more
- support for both exchange-native and normalized market data formats (unified format for accessing market data across all supported exchanges — normalized trades, order book and ticker data)
- seamless switching between real-time streaming and historical market data replay thanks to
async iterables
providing unified way of consuming data messages - transparent historical local data caching (cached data is stored on disk in compressed GZIP format and decompressed on demand when reading the data)
- support for top cryptocurrency exchanges: BitMEX, Deribit, Binance, Binance Futures, FTX, OKEx, Huobi Global, Huobi DM, bitFlyer, Bitstamp, Coinbase Pro, Kraken Futures, Gemini, Kraken, Bitfinex, Bybit, OKCoin, CoinFLEX and more
- automatic closed connections and stale connections reconnection logic for real-time streams
- combining multiple exchanges feeds into single one via
combine
helper function — synchronized historical market data replay and consolidated real-time data streaming from multiple exchanges - computing derived data locally like order book imbalance, customizable trade bars, book snapshots and more via
compute
helper function andcomputables
, e.g., volume based bars, top 20 levels order book snapshots taken every 10 ms etc - fast and lightweight architecture — low memory footprint and no heavy in-memory buffering
- built-in TypeScript support
Requires Node.js v12+ installed.
npm install tardis-dev --save
tardis-dev
lib uses debug package for verbose logging and debugging purposes that can be enabled via DEBUG
environment variable set to tardis-dev*
.
Simply change from require
const { replay, stream } = require('tardis-dev')
to ES Modules import
import { replay, stream } from 'tardis-dev'
to enjoy first class TypeScript typings.
See historical data details page to get detailed information about historical market data available for each exchange.
Replays historical market data messages for given replay options in exchange-native format. Historical market data is being fetched efficiently (in parallel) from the Tardis.dev HTTP API and cached locally. Returns
async iterable
.const { replay } = require('tardis-dev')
const messages = replay({
exchange: 'bitmex',
filters: [
{ channel: 'trade', symbols: ['XBTUSD'] },
{ channel: 'orderBookL2', symbols: ['XBTUSD'] }
],
from: '2019-05-01',
to: '2019-05-02'
})
for await (const message of messages) {
console.log(message)
}
stream(options) is the real-time counterpart of
replay
function, returning real-time market data in the same format.
name | type | default | |
exchange | string | - | |
filters | {channel:string, symbols?: string[]}[] | [] | optional filters of requested historical data feed - use getExchangeDetails function to get allowed channels and symbols ids for requested exchange |
from | string | - | |
to | string | - | |
skipDecoding | boolean | undefined | undefined | when set to true returns messages as buffers instead of decoding them to objects |
withDisconnects | boolean | undefined | undefined | when set to true returns message with value undefined for events when connection that was recording the historical data got disconnected |
apiKey | string | undefined | undefined |
type Message =
| {
localTimestamp: Date // local timestamp when message has been received
message: any // message in exchange-native data format
}
// when skipDecoding is set to true
| {
localTimestamp: Buffer
message: Buffer
}
// when withDisconnects is set to true whole message can be undefined (disconnect)
Message | undefined
{
localTimestamp: 2019-05-01T00:00:07.013Z,
message: {
table: 'orderBookL2',
action: 'update',
data: [ { symbol: 'XBTUSD', id: 8799474100, side: 'Buy', size: 167340 } ]
}
}
Replays historical market data messages for given replay options and normalizes messages using normalizers provided as rest arguments. Historical market data is being fetched efficiently (in parallel) from the Tardis.dev HTTP API and cached locally. Returns
async iterable
.const tardis = require('tardis-dev')
const { replayNormalized, normalizeTrades, normalizeBookChanges } = tardis
const messages = replayNormalized(
{
exchange: 'bitmex',
symbols: ['XBTUSD', 'ETHUSD'],
from: '2019-05-01',
to: '2019-05-02'
},
normalizeTrades,
normalizeBookChanges
)
for await (const message of messages) {
console.log(message)
}
streamNormalized(options, ...normalizers) is the real-time counterpart of
replayNormalized
function, returning real-time market data in the same format.
name | type | default | |
exchange | string | - | |
symbols | string[] | undefined | undefined | optional symbols for requested data feed - use getExchangeDetails function to get allowed symbols ids for requested exchange |
from | string | - | |
to | string | - | |
withDisconnectMessages | boolean | undefined | undefined | when set to true returns disconnect messages for events when connection that was recording the historical data got disconnected |
apiKey | string | undefined | undefined |
replayNormalized
function accepts any number of normalizers as rest parameters that map from exchange-native format to normalized data format. tardis-dev
ships with built in ones that normalize trades, order book and derivative ticker data but also allows adding custom ones.
Message types and formats depend on specific normalizers provided to
replayNormalized
function and are documented in detail in data normalization section.
{
type: 'book_change',
symbol: 'XBTUSD',
exchange: 'bitmex',
isSnapshot: false,
bids: [{ price: 5263.5, amount: 1780043 }],
asks: [],
timestamp: 2019-05-01T00:00:04.430Z,
localTimestamp: 2019-05-01T00:00:04.430Z
}
Streams real-time market data messages for given stream options in exchange-native format. It connects directly to exchanges WebSocket APIs and transparently restarts closed, broken or stale connections (open connections without data being send for specified amount of time). Returns
async iterable
.const { stream } = require('tardis-dev')
const messages = stream({
exchange: 'bitmex',
filters: [
{ channel: 'trade', symbols: ['XBTUSD'] },
{ channel: 'orderBookL2', symbols: ['XBTUSD'] }
]
})
for await (const message of messages) {
console.log(message)
}
replay(options) is the historical market data counterpart of
stream
function, returning historical market data in the same format.
name | type | default | |
exchange | string | - | |
filters | {channel:string, symbols?: string[]}[] | [] | optional filters of requested real-time data feed - use getExchangeDetails to get allowed channels and symbols ids for requested exchange |
skipDecoding | boolean | undefined | undefined | when set to true returns messages as buffers instead of decoding them to objects |
withDisconnects | boolean | undefined | undefined | when set to true returns message with value undefined for real-time stream disconnect events |
timeoutIntervalMS | number | 10000 | specifies time in milliseconds after which connection is restarted if no message has been received from the exchange |
onError | (err) => void | undefined | undefined | Optional callback invoked when real-time WebSocket connection error occurs, useful for custom error logging etc. |
type Message =
| {
localTimestamp: Date // local timestamp when message has been received
message: any // message in exchange-native data format
}
// when skipDecoding is set to true
| {
localTimestamp: Buffer
message: Buffer
}
// when withDisconnects is set to true whole message can be undefined
Message | undefined
{
localTimestamp: 2019-10-22T09:24:51.025Z,
message: {
table: 'orderBookL2',
action: 'update',
data: [
{ symbol: 'XBTUSD', id: 8799172400, side: 'Sell', size: 369501 }
]
}
}
Streams real-time market data messages for given stream options and normalizes messages using provided normalizers provided as rest arguments. It connects directly to exchanges WebSocket APIs and transparently restarts closed, broken or stale connections (open connections without data being send for specified amount of time). Returns
async iterable
.const tardis = require('tardis-dev')
const { streamNormalized, normalizeTrades, normalizeBookChanges } = tardis
const messages = streamNormalized(
{
exchange: 'bitmex',
symbols: ['XBTUSD', 'ETHUSD']
},
normalizeTrades,
normalizeBookChanges
)
for await (const message of messages) {
console.log(message)
}
replayNormalized(options) is the historical counterpart of
streamNormalized
function, returning historical market data in the same format.
name | type | default | |
exchange | string | - | |
symbols | string[] | undefined | undefined | instruments symbols for requested data feed |
withDisconnectMessages | boolean | undefined | undefined | |
timeoutIntervalMS | number | 10000 | specifies time in milliseconds after which connection is restarted if no message has been received from the exchange |
onError | ((err) => void) | undefined | undefined | Optional callback invoked when real-time WebSocket connection or mapping error occurs, useful for custom error logging etc. |
streamNormalized
function can accept any number of custom normalizers as rest parameters that map from exchange-native format to normalized data format. tardis-dev
ships with built in ones that normalize trades, order book and derivative ticker data but also allows adding custom ones.
Message types and formats depend on specific normalizers provided to
streamNormalized
function and are documented in detail in data normalization section.
{
type: 'trade',
symbol: 'XBTUSD',
exchange: 'bitmex',
id: 'b1f4b309-80e2-1ffb-340b-2f7576f6ef0d',
price: 7447.5,
amount: 100,
side: 'buy',
timestamp: 2019-10-24T13:28:07.867Z,
localTimestamp: 2019-10-24T13:28:07.938Z
}
This function doesn't affect real-time streaming functionality in any way, it's useful only for historical data replay.
When working with market data via
replay
and replayNormalized
functions by default only first day of each month of historical data is available for replay as well as locally cached historical data is stored in default location on disk (OS temp dir). Init
function allows providing apiKey
received via email after ordering historical market data access via Tardis.dev website as well as customcacheDir
. ApiKey
can also be provided directly via options of replay
and replayNormalized
functions - that overrides anything that was provided via init
.const { init } = require('tardis-dev')
init({
apiKey: 'YOUR API KEY',
cacheDir: 'CUSTOM CACHE DIR PATH'
})
name | type | defaults | |
apiKey | string | undefined | undefined | API key for Tardis.dev HTTP API - if not provided only first day of each month of historical data is accessible |
cacheDir | string | <os.tmpdir>/.tardis-cache | path to local dir that will be used as cache location - if not provided default temp dir for given OS will be used |
Given exchange id provides exchange details (available symbols, availability dates, available channels, pricing info etc) provided by exchanges/:exchange API endpoint.
const { getExchangeDetails } = require('tardis-dev')
const bitmexExchangeDetails = await getExchangeDetails('bitmex')
{
id: string
name: string
enabled: boolean
filterable: boolean
availableSince: string
availableSymbols: {
id: string
type: 'spot' | 'future' | 'perpetual' | 'option'
availableSince: string
availableTo?: string
}[]
availableChannels: string[]
incidentReports: {
from: string
to: string
status: string
details: string
}
}
Given
apiKey
provided as optional parameter or provided in init
function provides information about what historical data is available for it - exchanges, date ranges, symbols.const { getApiKeyAccessInfo } = require('tardis-dev')
const details = await getApiKeyAccessInfo('YOUR_API_KEY')
{
exchange: string
from: string
to: string
symbols: string[]
}[]
Clears local data cache dir.
const { clearCache } = require('tardis-dev')
await clearCache()
Data normalization allows consuming market data feeds from various exchanges in consistent format.
tardis-dev
has following built-in normalizers that can be provided to replayNormalized
or streamNormalized
functions:If you're interested in how exactly data is mapped from exchange-native format to normalized one, please follow code in
tardis-dev
GitHub repository for each exchange and if you determined that mapping should be done differently please read "modifying built-in and adding custom normalizers" section.const { streamNormalized, normalizeTrades, normalizeBookChanges,
normalizeDerivativeTickers } = require('tardis-dev')
// or replayNormalized to replay normalized historical data
const messages = streamNormalized(
{
exchange: 'deribit',
symbols: ['BTC-PERPETUAL']
},
normalizeTrades,
normalizeBookChanges,
normalizeDerivativeTickers
)
for await (const message of messages) {
if (message.type === 'book_change') {
// process normalized book change
}
if (message.type === 'trade') {
// process normalized trade
}
if (message.type === 'derivative_ticker') {
// process normalized derivative_ticker
}
}
When passed as an arg to
replayNormalized
or streamNormalized
function provides normalized trade
data for all supported exchanges. sample message
type definition
{
type: 'trade',
symbol: 'XBTUSD',
exchange: 'bitmex',
id: '282a0445-0e3a-abeb-f403-11003204ea1b',
price: 7996,
amount: 50,
side: 'sell',
timestamp: 2019-10-23T10:32:49.669Z,
localTimestamp: 2019-10-23T10:32:49.740Z
}
{
type: 'trade'
symbol: string
exchange: string
id: string | undefined
price: number
amount: number
side: 'buy' | 'sell' | 'unknown' // liquidity taker side (aggressor)
timestamp: Date
localTimestamp: Date
}
When passed as an arg to
replayNormalized
or streamNormalized
function provides normalized book_change
data for all supported exchanges. Provides initial L2 (market by price) order book snapshots (
isSnapshot=true
) plus incremental updates for each order book change. Please note that amount
is the updated amount at that price level, not a delta. An amount
of 0
indicates the price level can be removed.sample message
type definition
{
type: 'book_change',
symbol: 'XBTUSD',
exchange: 'bitmex',
isSnapshot: false,
bids: [],
asks: [{ price: 7985, amount: 283318 }],
timestamp: 2019-10-23T11:29:53.469Z,
localTimestamp: 2019-10-23T11:29:53.469Z
}
{
type: 'book_change'
symbol: string
exchange: string
isSnapshot: boolean
bids: { price: number; amount: number }[]
asks: { price: number; amount: number }[]
timestamp: Date
localTimestamp: Date
}
When passed as an arg to
replayNormalized
or streamNormalized
function provides normalized derivative_ticker
data for supported exchanges that trade derivative instruments. sample message
type definition
{
type: 'derivative_ticker',
symbol: 'BTC-PERPETUAL',
exchange: 'deribit',
lastPrice: 7987.5,
openInterest: 84129491,
fundingRate: -0.00001568,
indexPrice: 7989.28,
markPrice: 7987.56,
timestamp: 2019-10-23T11:34:29.302Z,
localTimestamp: 2019-10-23T11:34:29.416Z
}
{
type: 'derivative_ticker'
symbol: string
exchange: string
lastPrice: number | undefined
openInterest: number | undefined
fundingRate: number | undefined