tardis-dev
library provides convenient access to tick-level historical and real-time cryptocurrency market data both in exchange-native and normalized formats. Instead of callbacks it relies on async iteration (for await ...of) enabling composability features like seamless switching between real-time data streaming and historical data replay or computing derived data locally.async iterables
providing unified way of consuming data messages
combine
helper function — synchronized historical market data replay and consolidated real-time data streaming from multiple exchanges
compute
helper function and computables
, e.g., volume based bars, top 20 levels order book snapshots taken every 10 ms etc
tardis-dev
lib uses debug package for verbose logging and debugging purposes that can be enabled via DEBUG
environment variable set to tardis-dev*
.replay(options)
async iterable
.replay
function, returning real-time market data in the same format.getExchangeDetails
function to get allowed channels and symbols ids for requested exchangetrue
returns messages as buffers instead of decoding them to objectstrue
returns message with value undefined
for events when connection that was recording the historical data got disconnectedreplayNormalized(options, ...normalizers)
async iterable
.replayNormalized
function, returning real-time market data in the same format.getExchangeDetails
function to get allowed symbols ids for requested exchangetrue
returns disconnect
messages for events when connection that was recording the historical data got disconnectedreplayNormalized
function accepts any number of normalizers as rest parameters that map from exchange-native format to normalized data format. tardis-dev
ships with built in ones that normalize trades, order book and derivative ticker data but also allows adding custom ones.replayNormalized
function and are documented in detail in data normalization section.stream(options)
async iterable
.stream
function, returning historical market data in the same format.getExchangeDetails
to get allowed channels and symbols ids for requested exchangetrue
returns messages as buffers instead of decoding them to objectstrue
returns message with value undefined
for real-time stream disconnect eventsstreamNormalized(options, ...normalizers)
async iterable
.streamNormalized
function, returning historical market data in the same format.streamNormalized
function can accept any number of custom normalizers as rest parameters that map from exchange-native format to normalized data format. tardis-dev
ships with built in ones that normalize trades, order book and derivative ticker data but also allows adding custom ones.streamNormalized
function and are documented in detail in data normalization section.init(options)
replay
and replayNormalized
functions by default only first day of each month of historical data is available for replay as well as locally cached historical data is stored in default location on disk (OS temp dir). Init
function allows providing apiKey
received via email after ordering historical market data access via Tardis.dev website as well as customcacheDir
. ApiKey
can also be provided directly via options of replay
and replayNormalized
functions - that overrides anything that was provided via init
.temp
dir for given OS will be usedgetExchangeDetails(exchange)
getExchangeDetails
getApiKeyAccessInfo(apiKey?)
apiKey
provided as optional parameter or provided in init
function provides information about what historical data is available for it - exchanges, date ranges, symbols.getApiKeyAccessInfo()
clearCache()
tardis-dev
has following built-in normalizers that can be provided to replayNormalized
or streamNormalized
functions:tardis-dev
GitHub repository for each exchange and if you determined that mapping should be done differently please read "modifying built-in and adding custom normalizers" section.normalizeTrades
replayNormalized
or streamNormalized
function provides normalized trade
data for all supported exchanges. normalizeBookChanges
replayNormalized
or streamNormalized
function provides normalized book_change
data for all supported exchanges. isSnapshot=true
) plus incremental updates for each order book change. Please note that amount
is the updated amount at that price level, not a delta. An amount
of 0
indicates the price level can be removed.normalizeDerivativeTickers
replayNormalized
or streamNormalized
function provides normalized derivative_ticker
data for supported exchanges that trade derivative instruments. disconnect
messagereplayNormalized
or streamNormalized
functions options have withDisconnectMessages
flag set to true
and disconnect event occurred (eg.: WebSocket connection close) then disconnect
message is being returned.tardis-dev
data normalization is implemented via normalize
factory functions provided to replayNormalized
and streamNormalized
functions. This design gives lots of flexibility by allowing replacing, extending and modifying built-in normalizers or adding new ones for new normalized data types without the need of forking the whole library. replayNormalized
and streamNormalized
functions needs to have following signature:Exchange
is an exchange id for which mapper object needs to be returned for, localTimestamp
is a date for which mapper is created (and is created after each disconnection). In most cases localTimestamp
is not necessary for anything, but in certain cases like for example exchange API change it can be used to switch to different mapping logic like using new data channel that wasn't available until certain date.Mapper
object has following signature:Mapper
objects with clean state if required (stateful mapping like BitMEX order book data that needs to persist state of mapping between price level id and price level and needs to 'reset' for each new connection). If mapper object is stateful it's required to always return new clean state object from normalize factory function or reset it's state in one way or another.iterable iterator
of Mapper.map
method is expected to have a shape that has at least fields as described in normalized data type section below to play well with other tardis-dev
functions like combine
or compute
.normalizeLiquidations
normalizernormalizeLiquidations
function that normalizes liquidations data for deribit
exchange. Implementations for for other exchanges are left as an exercise for the reader.normalizeLiquidations
deribitLiquidations
mapper and normalizeLiquidations