Downloadable CSV files

Quick start

CSV datasets are available via dedicated datasets API that allows downloading tick level incremental order book L2 updates, order book snapshots, trades, options chains, quotes, derivative tickers and liquidations data. For ongoing data, CSV datasets for a given day are available on the next day around 06:00 UTC.

CSV datasets are exported from exchanges' real-time WebSocket feeds data we collected (and also provide via our API as historical data in exchange-native format).

Historical datasets for the first day of each month are available to download without API key. Our Node.js and Python clients have built-in functions to efficiently download whole date range of data.

Python

# pip install tardis-dev
# requires Python >=3.6
from tardis_dev import datasets

datasets.download(
    exchange="deribit",
    data_types=[
        "incremental_book_L2",
        "trades",
        "quotes",
        "derivative_ticker",
        "book_snapshot_25",
        "liquidations"
    ],
    from_date="2019-11-01",
    to_date="2019-11-02",
    symbols=["BTC-PERPETUAL", "ETH-PERPETUAL"],
    api_key="YOUR API KEY (optionally)",
)

See full example that shows all available download options (download path customization, filenames conventions and more).

Node.js

// npm install [email protected]
// requires node version >=12
const { downloadDatasets } = require('tardis-dev')

;(async () => {
  await downloadDatasets({
    exchange: 'deribit',
    dataTypes: [
      'incremental_book_L2',
      'trades',
      'quotes',
      'derivative_ticker',
      'book_snapshot_25',
      'liquidations'
    ],
    from: '2019-11-01',
    to: '2019-11-02',
    symbols: ['BTC-PERPETUAL', 'ETH-PERPETUAL'],
    apiKey: 'YOUR API KEY (optionally)'
  })
})()

See full example that shows all available download options (download path customization, filenames conventions and more).

cURL

curl -o deribit_trades_2019-11-01_BTC-PERPETUAL.csv.gz https://datasets.tardis.dev/v1/deribit/trades/2019/11/01/BTC-PERPETUAL.csv.gz

See datasets API reference.

CSV format details

• columns delimiter: , (comma)

• new line marker: \n (LF)

• decimal mark: . (dot)

• date time format: microseconds since epoch (https://www.epochconverter.com/)

• date time timezone: UTC

Data types

• incremental_book_L2

Incremental order book L2 updates collected from exchanges' real-time WebSocket order book L2 data feeds - data as deep and granular as underlying real-time data source, please see FAQ: What is the maximum order book depth available for each supported exchange? for more details.

Learn in more detail how incremental_book_l2 CSV dataset is built from real-time data.

As exchanges real-time feeds usually publish multiple order book levels updates via single message you can recognize that by grouping rows by local_timestamp field if needed.

If you have any doubts how to correctly reconstruct full order book state from incremental_book_L2 CSV dataset, please see this answer or contact us.

In case you only need order book data for top 25 or top 5 levels, we do provide datasets with already reconstructed snapshots for every update for those. See book_snapshot_25 and book_snapshot_5.

CSV incremental_book_L2 schema

column name	description
exchange	exchange id, one of https://api.tardis.dev/v1/exchanges ([].id field)
symbol	instrument symbol as provided by exchange (always uppercase)
timestamp	timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback
local_timestamp	message arrival timestamp in microseconds since epoch
is_snapshot	possible values: true - if update was a part of initial order book snapshot false - if update was not a part of initial order book snapshot If last update was not a snapshot and current one is, then existing order book state must be discarded (all existing levels removed)
side	determines to which side of the order book update belongs to: bid - bid side of the book, buy orders ask - ask side of the book, sell orders
price	price identifying book level being updated
amount	updated price level amount as provided by exchange, not a delta - an amount of 0 indicates that the price level can be removed

dataset preview

exchange	symbol	timestamp	local_timestamp	is_snapshot	side	price	amount
deribit	BTC-PERPETUAL	1585699209920000	1585699209934201	false	ask	6443.5	38640
deribit	BTC-PERPETUAL	1585699209947000	1585699209957629	false	bid	6311.5	0
deribit	BTC-PERPETUAL	1585699209950000	1585699209963464	false	ask	6428	13210
deribit	BTC-PERPETUAL	1585699209967000	1585699209979152	false	bid	6311.5	750
deribit	BTC-PERPETUAL	1585699209970000	1585699209983585	false	bid	6327	16010
deribit	BTC-PERPETUAL	1585699209970000	1585699209983585	false	bid	6325	210530
deribit	BTC-PERPETUAL	1585699209972000	1585699209983691	false	bid	6351	810
deribit	BTC-PERPETUAL	1585699209972000	1585699209983691	false	bid	6352.5	18830
deribit	BTC-PERPETUAL	1585699209974000	1585699209983703	false	ask	6492	100

https://datasets.tardis.dev/v1/deribit/incremental_book_L2/2020/04/01/BTC-PERPETUAL.csv.gz

Deribit BTC-PERPETUAL incremental order book L2 updates for 2020-04-01

https://datasets.tardis.dev/v1/deribit/incremental_book_L2/2020/09/01/FUTURES.csv.gz

Deribit FUTURES instruments incremental order book L2 updates for 2020-09-01

• book_snapshot_25

Tick-level order book snapshots reconstructed from exchanges' real-time WebSocket order book L2 data feeds. Each row represents top 25 levels from each side of the limit order book book and was recorded every time any of the tracked bids/asks top 25 levels have changed.

CSV book_snapshot_25 schema

column name	description
exchange	exchange id, one of https://api.tardis.dev/v1/exchanges ([].id field)
symbol	instrument symbol as provided by exchange (always uppercase)
timestamp	timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback
local_timestamp	message arrival timestamp in microseconds since epoch
asks[0..24].price	top 25 asks prices in ascending order, empty if there aren't enough price levels available in the order book or provided by the exchange
asks[0..24].amount	top 25 asks amounts in ascending order, empty if there aren't enough price levels available in the order book or provided by the exchange
bids[0..24].price	top 25 bids prices in descending order, empty if there aren't enough price levels available in the order book or provided by the exchange
bids[0..24].amount	top 25 bids amounts in descending order, empty if there aren't enough price levels available in the order book or provided by the exchange

dataset preview

exchange	symbol	timestamp	local_timestamp	asks[0].price	asks[0].amount	bids[0].price	bids[0].amount	asks[1].price	asks[1].amount	bids[1].price	bids[1].amount	asks[2].price	asks[2].amount	bids[2].price	bids[2].amount	asks[3].price	asks[3].amount	bids[3].price	bids[3].amount	asks[4].price	asks[4].amount	bids[4].price	bids[4].amount	asks[5].price	asks[5].amount	bids[5].price	bids[5].amount	asks[6].price	asks[6].amount	bids[6].price	bids[6].amount	asks[7].price	asks[7].amount	bids[7].price	bids[7].amount	asks[8].price	asks[8].amount	bids[8].price	bids[8].amount	asks[9].price	asks[9].amount	bids[9].price	bids[9].amount	asks[10].price	asks[10].amount	bids[10].price	bids[10].amount	asks[11].price	asks[11].amount	bids[11].price	bids[11].amount	asks[12].price	asks[12].amount	bids[12].price	bids[12].amount	asks[13].price	asks[13].amount	bids[13].price	bids[13].amount	asks[14].price	asks[14].amount	bids[14].price	bids[14].amount	asks[15].price	asks[15].amount	bids[15].price	bids[15].amount	asks[16].price	asks[16].amount	bids[16].price	bids[16].amount	asks[17].price	asks[17].amount	bids[17].price	bids[17].amount	asks[18].price	asks[18].amount	bids[18].price	bids[18].amount	asks[19].price	asks[19].amount	bids[19].price	bids[19].amount	asks[20].price	asks[20].amount	bids[20].price	bids[20].amount	asks[21].price	asks[21].amount	bids[21].price	bids[21].amount	asks[22].price	asks[22].amount	bids[22].price	bids[22].amount	asks[23].price	asks[23].amount	bids[23].price	bids[23].amount	asks[24].price	asks[24].amount	bids[24].price	bids[24].amount
deribit	BTC-PERPETUAL	1599868800206000	1599868800253274	10396	48050	10395.5	18220	10396.5	22220	10395	16570	10397	100	10394.5	22630	10397.5	8360	10394	16670	10398	1500	10393.5	16570	10398.5	13210	10393	5600	10399.5	60070	10392.5	20500	10400	5100	10392	30	10400.5	5140	10391.5	75780	10401	13040	10391	12110	10401.5	2250	10390.5	280	10402	9150	10390	52680	10402.5	119390	10389.5	18240	10403	23070	10389	73010	10403.5	53930	10388.5	67500	10404	43590	10388	313140	10404.5	271050	10387.5	280	10405	73710	10387	9840	10405.5	32480	10386.5	104570	10406	41220	10386	269050	10406.5	20400	10385.5	21840	10407	45460	10385	79000	10407.5	69630	10384.5	220	10408	22230	10384	71440	10408.5	30840	10383.5	44740
deribit	BTC-PERPETUAL	1599868800280000	1599868800310441	10396	48050	10395.5	18220	10396.5	22220	10395	16570	10397	100	10394.5	22630	10397.5	8360	10394	16670	10398	1500	10393.5	16570	10398.5	13210	10393	5600	10399.5	60070	10392.5	20500	10400	5100	10392	30	10400.5	5140	10391.5	75780	10401	13040	10391	12110	10401.5	2250	10390.5	280	10402	9150	10390	52680	10402.5	119390	10389.5	18240	10403	23070	10389	73010	10403.5	53930	10388.5	67500	10404	43590	10388	313140	10404.5	271050	10387.5	280	10405	73710	10387	9850	10405.5	32480	10386.5	104570	10406	41220	10386	269050	10406.5	20400	10385.5	21840	10407	45460	10385	79000	10407.5	69630	10384.5	220	10408	22230	10384	71440	10408.5	30840	10383.5	44740
deribit	BTC-PERPETUAL	1599868814801000	1599868814817631	10398.5	20	10398	7400	10399	4890	10397.5	17680	10399.5	520	10396.5	17680	10400	1700	10396	30280	10400.5	3010	10395.5	44110	10401	40	10395	20080	10401.5	2570	10394.5	91410	10402	400	10394	97570	10402.5	50530	10393.5	27510	10403	9960	10393	3330	10403.5	54250	10392.5	200	10404	40	10392	20400	10404.5	10	10391.5	75650	10405	93470	10391	9580	10405.5	32540	10390.5	260040	10406	26130	10390	310	10406.5	9670	10389.5	21210	10407	1180	10389	87320	10407.5	89030	10388.5	61140	10408	54860	10388	283120	10408.5	42430	10387.5	10680	10409	260680	10387	11400	10409.5	19220	10386.5	92470	10410	94970	10386	49640	10410.5	50	10385.5	6420
deribit	BTC-PERPETUAL	1599868814809000	1599868814817632	10398.5	20	10398	7400	10399	4890	10397.5	17680	10399.5	520	10396.5	17680	10400	1700	10396	30280	10400.5	3010	10395.5	44110	10401	40	10395	20080	10401.5	2570	10394.5	91410	10402	400	10394	97570	10402.5	50530	10393.5	27510	10403	9960	10393	3330	10403.5	54900	10392.5	200	10404	40	10392	20400	10404.5	10	10391.5	75650	10405	93470	10391	9580	10405.5	32540	10390.5	260040	10406	26130	10390	310	10406.5	9670	10389.5	21210	10407	1180	10389	87320	10407.5	89030	10388.5	61140	10408	54860	10388	283120	10408.5	42430	10387.5	10680	10409	260680	10387	11400	10409.5	19220	10386.5	92470	10410	94970	10386	49640	10410.5	50	10385.5	6420
deribit	BTC-PERPETUAL	1599868815411000	1599868815414125	10399	4910	10398	25080	10399.5	20	10397.5	17680	10400	2200	10396.5	17680	10400.5	2910	10396	31780	10401	40	10395.5	44110	10401.5	570	10395	20050	10402	500	10394.5	91440	10402.5	52990	10394	98510	10403	3500	10393.5	26570	10403.5	45100	10393	3330	10404	9190	10392.5	470	10404.5	10	10392	18300	10405	70030	10391.5	85130	10405.5	60800	10391	8640	10406	26130	10390.5	260040	10406.5	9270	10390	22530	10407	240	10389.5	14030	10407.5	89970	10389	65120	10408	23640	10388.5	72380	10408.5	62090	10388	283120	10409	260680	10387.5	10280	10409.5	18150	10387	11400	10410	94970	10386.5	123630	10410.5	50	10386	8470	10411	28210	10385.5	6420
deribit	BTC-PERPETUAL	1599868815411000	1599868815419035	10399	4910	10398	25080	10399.5	20	10397.5	17680	10400	2200	10396.5	17680	10400.5	2910	10396	31780	10401	40	10395.5	44110	10401.5	570	10395	20050	10402	500	10394.5	91440	10402.5	52990	10394	98510	10403	3500	10393.5	26570	10403.5	45100	10393	3330	10404	9190	10392.5	470	10404.5	10	10392	18300	10405	70030	10391.5	85130	10405.5	60800	10391	8640	10406	26130	10390.5	260040	10406.5	17270	10390	22530	10407	240	10389.5	14030	10407.5	89970	10389	65120	10408	23640	10388.5	72380	10408.5	62090	10388	283120	10409	260680	10387.5	10280	10409.5	18150	10387	11400	10410	94970	10386.5	123630	10410.5	50	10386	8470	10411	28210	10385.5	6420
deribit	BTC-PERPETUAL	1599868907943000	1599868907946933	10398	4600	10397.5	73090	10399.5	10	10397	24630	10400	3300	10396.5	22770	10400.5	10270	10396	3130	10401	25390	10395.5	5000	10401.5	119790	10395	9060	10402	8510	10394.5	17910	10402.5	8180	10394	138990	10403	10000	10393.5	33080	10404	7960	10393	3020	10404.5	15130	10392.5	8130	10405	128930	10392	100920	10405.5	109560	10391.5	83330	10406	8610	10391	32220	10406.5	34890	10390.5	278270	10407	44440	10390	47980	10407.5	102620	10389.5	292240	10408	20660	10389	65100	10408.5	175160	10388.5	790	10409	7660	10388	55720	10409.5	308550	10387.5	31440	10410	138130	10387	3830	10410.5	15940	10386.5	109470	10411	2610	10386	31560	10411.5	3780	10385.5	3450
deribit	BTC-PERPETUAL	1599868907944000	1599868907953129	10398	4600	10397.5	73090	10399.5	10	10397	24630	10400	3300	10396.5	22770	10400.5	10270	10396	3130	10401	25390	10395.5	5000	10401.5	119790	10395	1060	10402	8510	10394.5	17910	10402.5	8180	10394	146990	10403	10000	10393.5	33080	10404	7960	10393	3020	10404.5	15130	10392.5	8130	10405	128930	10392	100920	10405.5	109560	10391.5	83330	10406	8610	10391	32220	10406.5	34890	10390.5	278270	10407	44440	10390	47980	10407.5	102620	10389.5	292240	10408	20660	10389	65100	10408.5	175160	10388.5	790	10409	7660	10388	55720	10409.5	308550	10387.5	31440	10410	138130	10387	3830	10410.5	15940	10386.5	109470	10411	2610	10386	31560	10411.5	3780	10385.5	3450
deribit	BTC-PERPETUAL	1599868907993000	1599868907997022	10398	4600	10397.5	73090	10399.5	2010	10397	24630	10400	3300	10396.5	22770	10400.5	8270	10396	3130	10401	25390	10395.5	5000	10401.5	119790	10395	1060	10402	8510	10394.5	17910	10402.5	8180	10394	146990	10403	10000	10393.5	33080	10404	7960	10393	3020	10404.5	15130	10392.5	8130	10405	128930	10392	100920	10405.5	109560	10391.5	83330	10406	8610	10391	32220	10406.5	34890	10390.5	278270	10407	44440	10390	47980	10407.5	102620	10389.5	292240	10408	20660	10389	65100	10408.5	175160	10388.5	790	10409	7660	10388	55720	10409.5	308550	10387.5	31440	10410	138130	10387	3830	10410.5	15940	10386.5	109470	10411	2610	10386	31560	10411.5	3780	10385.5	3450

https://datasets.tardis.dev/v1/bitmex/book_snapshot_25/2020/09/01/XBTUSD.csv.gz

BitMEX XBTUSD top 25 levels order book snapshots for 2020-09-01

https://datasets.tardis.dev/v1/binance-futures/book_snapshot_25/2020/09/01/BTCUSDT.csv.gz

Binance USDT Futures BTCUSDT top 25 levels order book snapshots for 2020-09-01

• book_snapshot_5

Tick-level order book snapshots reconstructed from exchanges' real-time WebSocket order book L2 data feeds. Each row represents top 5 levels from each side of the limit order book book and was recorded every time any of the tracked bids/asks top 5 levels have changed.

CSV book_snapshot_5 schema

column name	description
exchange	exchange id, one of https://api.tardis.dev/v1/exchanges ([].id field)
symbol	instrument symbol as provided by exchange (always uppercase)
timestamp	timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback
local_timestamp	message arrival timestamp in microseconds since epoch
asks[0..4].price	top 5 asks prices in ascending order, empty if there aren't enough price levels available in the order book or provided by the exchange
asks[0..4].amount	top 5 asks amounts in ascending order, empty if there aren't enough price levels available in the order book or provided by the exchange
bids[0..4].price	top 5 bids prices in descending order, empty if there aren't enough price levels available in the order book or provided by the exchange
bids[0..4].amount	top 5 bids amounts in descending order, empty if there aren't enough price levels available in the order book or provided by the exchange

dataset preview

exchange	symbol	timestamp	local_timestamp	asks[0].price	asks[0].amount	bids[0].price	bids[0].amount	asks[1].price	asks[1].amount	bids[1].price	bids[1].amount	asks[2].price	asks[2].amount	bids[2].price	bids[2].amount	asks[3].price	asks[3].amount	bids[3].price	bids[3].amount	asks[4].price	asks[4].amount
bitmex	XBTUSD	1598918402683390	1598918402683390	11658	1399982	11657.5	2293327	11658.5	82328	11657	37555	11659	3001	11656.5	110647	11659.5	10843	11656	10063	11660	2522
bitmex	XBTUSD	1598918403229829	1598918403229829	11658	1399982	11657.5	2293327	11658.5	82328	11657	37555	11659	3001	11656.5	110647	11659.5	10835	11656	10063	11660	2522
bitmex	XBTUSD	1598918403232925	1598918403232925	11658	1399982	11657.5	2295327	11658.5	82328	11657	37555	11659	3001	11656.5	110647	11659.5	10835	11656	10063	11660	2522
bitmex	XBTUSD	1598918403253585	1598918403253585	11658	1399982	11657.5	2295256	11658.5	82328	11657	37555	11659	3001	11656.5	110647	11659.5	10835	11656	10063	11660	2522
bitmex	XBTUSD	1598918403256460	1598918403256460	11658	1399982	11657.5	2294159	11658.5	82328	11657	37555	11659	3001	11656.5	110647	11659.5	10835	11656	10063	11660	2522
bitmex	XBTUSD	1598947264502293	1598947264502293	11950	730542	11949.5	1100309	11950.5	30454	11949	454098	11951	72967	11948.5	2519605	11951.5	48967	11948	97449	11952	55623
bitmex	XBTUSD	1598947264505452	1598947264505452	11950	730542	11949.5	1100309	11950.5	30454	11949	454098	11951	72967	11948.5	2509605	11951.5	48967	11948	97449	11952	55623
bitmex	XBTUSD	1598947264510015	1598947264510015	11950	730542	11949.5	1100975	11950.5	30454	11949	454098	11951	72967	11948.5	2509605	11951.5	48967	11948	97449	11952	55623
bitmex	XBTUSD	1598947264510024	1598947264510024	11950	730542	11949.5	1250975	11950.5	30454	11949	454098	11951	72967	11948.5	2509605	11951.5	48967	11948	97449	11952	55623

https://datasets.tardis.dev/v1/bitmex/book_snapshot_5/2020/09/01/XBTUSD.csv.gz

BitMEX XBTUSD top 5 levels order book snapshots for 2020-09-01

https://datasets.tardis.dev/v1/binance-futures/book_snapshot_5/2020/09/01/BTCUSDT.csv.gz

Binance USDT Futures BTCUSDT top 5 levels order book snapshots for 2020-09-01

• trades

Individual trades data collected from exchanges' real-time WebSocket trades data feeds.

CSV trades schema

column name	description
exchange	exchange id, one of https://api.tardis.dev/v1/exchanges ([].id field)
symbol	instrument symbol as provided by exchange (always uppercase)
timestamp	timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback
local_timestamp	message arrival timestamp in microseconds since epoch
id	trade id as provided by exchange, empty if exchange does not provide one - different exchanges provide id's as numeric values, GUID's or other strings, and some do not provide that information at all
side	liquidity taker side (aggressor), possible values: buy - liquidity taker was buying sell - liquidity taker was selling unknown - exchange did not provide that information
price	trade price as provided by exchange
amount	trade amount as provided by exchange

dataset preview

exchange	symbol	timestamp	local_timestamp	id	side	price	amount
bitmex	XBTUSD	1585699202957000	1585699203089980	d20...	buy	6425.5	12
bitmex	XBTUSD	1585699202980000	1585699203095276	619...	sell	6425	150
bitmex	XBTUSD	1585699203002000	1585699203099299	751...	sell	6425	25
bitmex	XBTUSD	1585699203092000	1585699203122233	3c1...	buy	6425.5	1
bitmex	XBTUSD	1585699203092000	1585699203122233	b9b...	buy	6425.5	1
bitmex	XBTUSD	1585699203092000	1585699203122233	433...	buy	6425.5	1
bitmex	XBTUSD	1585699203092000	1585699203122233	d16...	buy	6425.5	1
bitmex	XBTUSD	1585699203092000	1585699203122233	402...	buy	6425.5	1
bitmex	XBTUSD	1585699203092000	1585699203122233	2f8...	buy	6425.5	1

https://datasets.tardis.dev/v1/bitmex/trades/2020/03/01/XBTUSD.csv.gz

Bitmex XBTUSD trades for 2020-03-01 dataset sample

https://datasets.tardis.dev/v1/okex-futures/trades/2020/03/01/FUTURES.csv.gz

OKEx Futures FUTURES instruments trades for 2020-03-01 dataset sample

• options_chain

Tick-level options summary info (strike prices, expiration dates, open interest, implied volatility, greeks etc.) for all active options instruments collected from exchanges' real-time WebSocket options tickers data feeds. Options chain data is available for Deribit (sourced from ticker channel) and OKEx Options (sourced from option/summary and index/ticker channels).

For options_chain data type only 'OPTIONS' symbol is available (one file per day for all options instruments).

CSV options_chain schema

column name	description
exchange	exchange id, one of https://api.tardis.dev/v1/exchanges ([].id field)
symbol	instrument symbol as provided by exchange (always uppercase)
timestamp	ticker timestamp provided by exchange in microseconds since epoch
local_timestamp	ticker message arrival timestamp in microseconds since epoch
type	option type, possible values: put call
strike_price	option strike price
expiration	option expiration date in microseconds since epoch
open_interest	current open interest, empty is exchange does not provide one
last_price	price of the last trade, empty if there weren't any trades yet
bid_price	current best bid price, empty if there aren't any bids
bid_amount	current best bid amount, empty if there aren't any bids
bid_iv	implied volatility for best bid, empty if there aren't any bids
ask_price	current best ask price, empty if there aren't any asks
ask_amount	current best ask amount, empty if there aren't any asks
ask_iv	implied volatility for best ask, empty if there aren't any asks
mark_price	mark price, empty is exchange does not provide one
mark_iv	implied volatility for mark price, empty is exchange does not provide one
underlying_index	underlying index name that option contract is based upon
underlying_price	underlying price, empty is exchange does not provide one
delta	delta value for the option, empty is exchange does not provide one
gamma	gamma value for the option, empty is exchange does not provide one
vega	vega value for the option, empty is exchange does not provide one
theta	theta value for the option, empty is exchange does not provide one
rho	rho value for the option, empty is exchange does not provide one

dataset preview

exchange	symbol	timestamp	local_timestamp	type	strike_price	expiration	open_interest	last_price	bid_price	bid_amount	bid_iv	ask_price	ask_amount	ask_iv	mark_price	mark_iv	underlying_index	underlying_price	delta	gamma	vega	theta	rho
deribit	BTC-9JUN20-9875-P	1591574399413000	1591574400196008	put	9875	1591689600000000	0.1	0.0295	0.0205	15.0	55.91	0.0235	15.0	68.94	0.02210436	62.89	SYN.BTC-9JUN20	9756.36	-0.61752	0.00103	2.24964	-53.05655	-0.22796
deribit	BTC-9JUN20-9875-P	1591574404454000	1591574404473112	put	9875	1591689600000000	0.1	0.0295	0.0205	15.0	55.91	0.0235	15.0	68.94	0.02209480	62.86	SYN.BTC-9JUN20	9756.37	-0.61757	0.00103	2.24954	-53.02754	-0.22798
deribit	BTC-9JUN20-9875-C	1591574397505000	1591574400196010	call	9875	1591689600000000	44.3	0.0080	0.0095	0.5	61.00	0.0105	20.0	65.33	0.00992836	62.87	SYN.BTC-9JUN20	9756.25	0.38232	0.00103	2.24933	-53.03038	0.13272
deribit	BTC-9JUN20-9750-C	1591574399414000	1591574400196011	call	9750	1591689600000000	30.5	0.0145	0.0145	0.3	58.80	0.0160	20.0	65.02	0.01527998	62.05	SYN.BTC-9JUN20	9756.36	0.51442	0.00109	2.35092	-54.69903	0.17789
deribit	BTC-9JUN20-9750-P	1591574397562000	1591574400196012	put	9750	1591689600000000	0.8	0.0185	0.0140	0.3	59.40	0.0155	0.3	65.63	0.01464260	62.06	SYN.BTC-9JUN20	9756.25	-0.48570	0.00109	2.35092	-54.70775	-0.17832
deribit	BTC-9JUN20-9625-P	1591574397824000	1591574400197202	put	9625	1591689600000000	9.5	0.0130	0.0090	0.4	61.64	0.0105	0.4	68.29	0.00975848	65.01	SYN.BTC-9JUN20	9756.25	-0.35780	0.00097	2.20136	-53.66975	-0.13100
deribit	BTC-9JUN20-9625-C	1591574397359000	1591574400197208	call	9625	1591689600000000	18.0	0.0220	0.0215	15.0	57.39	0.0235	20.0	66.31	0.02320750	65.02	SYN.BTC-9JUN20	9756.18	0.64212	0.00097	2.20150	-53.67532	0.22058
deribit	BTC-9JUN20-9500-P	1591574397940000	1591574400197209	put	9500	1591689600000000	51.5	0.0065	0.0060	0.5	66.31	0.0065	17.1	68.91	0.00625625	67.64	SYN.BTC-9JUN20	9756.30	-0.25091	0.00080	1.87744	-47.62196	-0.09165
deribit	BTC-9JUN20-9500-C	1591574399413000	1591574400197211	call	9500	1591689600000000	43.8	0.0165	0.0315	20.0	62.19	0.0350	0.4	80.06	0.03252520	67.64	SYN.BTC-9JUN20	9756.36	0.74914	0.00080	1.87725	-47.61618	0.25540

https://datasets.tardis.dev/v1/deribit/options_chain/2020/03/01/OPTIONS.csv.gz

Deribit options chain for 2020-03-01

https://datasets.tardis.dev/v1/okex-options/options_chain/2020/03/01/OPTIONS.csv.gz

OKEx options chain for 2020-03-01

• quotes

Top of the book (best bid/ask) data reconstructed from exchanges' real-time WebSocket order book L2 data feeds. - best bid/ask recorded every time top of the book has changed. We on purpose choose this solution over native exchanges real-time quotes feeds as those vary a lot between exchanges, can be throttled, some are absent at all, often are delayed and published in batches in comparison to more granular L2 updates which are the basis for our quotes dataset.

CSV quotes schema

column name	description
exchange	exchange id, one of https://api.tardis.dev/v1/exchanges ([].id field)
symbol	instrument symbol as provided by exchange (always uppercase)
timestamp	timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback
local_timestamp	message arrival timestamp in microseconds since epoch
ask_amount	best ask amount as provided by exchange, empty if there aren't any asks
ask_price	best ask price as provided by exchange, empty if there aren't any asks
bid_price	best bid price as provided by exchange, empty if there aren't any bids
bid_amount	best bid amount as provided by exchange, empty if there aren't any bids

dataset preview

exchange	symbol	timestamp	local_timestamp	ask_amount	ask_price	bid_price	bid_amount
huobi-dm-swap	BTC-USD	1585699201147000	1585699201270777	86	6423	6422.9	112
huobi-dm-swap	BTC-USD	1585699201175000	1585699201292111	86	6423	6422.9	114
huobi-dm-swap	BTC-USD	1585699201257000	1585699201373479	84	6423	6422.9	219
huobi-dm-swap	BTC-USD	1585699201279000	1585699201495667	64	6423	6422.9	219
huobi-dm-swap	BTC-USD	1585699201295000	1585699201495715	64	6423	6422.9	229
huobi-dm-swap	BTC-USD	1585699201447000	1585699201564788	2	6423	6422.9	229
huobi-dm-swap	BTC-USD	1585699201556000	1585699201677770	64	6423	6422.9	229
huobi-dm-swap	BTC-USD	1585699201668000	1585699201784213	64	6423	6422.9	235
huobi-dm-swap	BTC-USD	1585699201747000	1585699201865051	2	6423	6422.9	235

https://datasets.tardis.dev/v1/huobi-dm-swap/quotes/2020/05/01/BTC-USD.csv.gz

Huobi DM Swap BTC-USD quotes for 2020-05-01

https://datasets.tardis.dev/v1/options/quotes/2020/05/01/OPTIONS.csv.gzdatasets.tardis.dev

Deribit OPTIONS instruments quotes for 2020-05-01

book_ticker

• derivative_ticker

Derivative instrument ticker info (open interest, funding, mark price, index price) collected from exchanges' real-time WebSocket instruments & tickers data feeds. Anytime any of the tracked values has changed data was added to final dataset.

CSV derivative_ticker schema

column name	description
exchange	exchange id, one of https://api.tardis.dev/v1/exchanges ([].id field)
symbol	instrument symbol as provided by exchange (always uppercase)
timestamp	timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback
local_timestamp	message arrival timestamp in microseconds since epoch
funding_timestamp	timestamp of the next funding event in microseconds since epoch, empty if exchange does not provide one
funding_rate	funding rate that will take effect on the next funding event at funding timestamp, for some exchanges it's fixed, for other it fluctuates, empty if exchange does not provide one
predicted_funding_rate	estimated predicted funding rate for the next after closest funding event, empty if exchange does not provide one
open_interest	current open interest, empty if exchange does not provide one
last_price	last instrument price, empty if exchange does not provide one
index_price	index price of the instrument, empty if exchange does not provide one
mark_price	mark price of the instrument, empty if exchange does not provide one

dataset preview

1	exchange	symbol	timestamp	local_timestamp	funding_timestamp	funding_rate	predicted_funding_rate	open_interest	last_price	index_price	mark_price
2	bitmex	ETHUSD	1585699199651000	1585699202577291	1585713600000000	0.0001	0.001654	45921455	133.25	133.14	133.15
3	bitmex	ETHUSD	1585699200000000	1585699204834359	1585713600000000	0.0001	0.001654	45921455	133.25	133.12	133.13
4	bitmex	ETHUSD	1585699202925000	1585699205076090	1585713600000000	0.0001	0.001654	45921455	133.3	133.12	133.13
5	bitmex	ETHUSD	1585699202925000	1585699205090339	1585713600000000	0.0001	0.001654	45883853	133.3	133.12	133.13
6	bitmex	ETHUSD	1585699203465000	1585699205274555	1585713600000000	0.0001	0.001654	45883853	133.25	133.12	133.13
7	bitmex	ETHUSD	1585699204439000	1585699205951209	1585713600000000	0.0001	0.001654	45883853	133.15	133.12	133.13
8	bitmex	ETHUSD	1585699205000000	1585699206389317	1585713600000000	0.0001	0.001654	45883853	133.15	133.09	133.1
9	bitmex	ETHUSD	1585699207279000	1585699207490211	1585713600000000	0.0001	0.001654	45883853	133.2	133.09	133.1
10	bitmex	ETHUSD	1585699207279000	1585699208084951	1585713600000000	0.0001	0.001654	45867677	133.2	133.09	133.1

https://datasets.tardis.dev/v1/bitmex/derivative_ticker/2019/04/01/ETHUSD.csv.gz

BitMEX ETHUSD derivative ticker for 2019-04-01

https://datasets.tardis.dev/v1/ftx/derivative_ticker/2019/04/01/PERPETUALS.csv.gzdatasets.tardis.dev

FTX PERPETUALS instruments derivative ticker for 2019-04-01

• liquidations

Liquidations data collected from exchanges' real-time WebSocket data feeds were available.

See details which exchanges support it and since when.

CSV liquidations schema

column name	description
exchange	exchange id, one of https://api.tardis.dev/v1/exchanges ([].id field)
symbol	instrument symbol as provided by exchange (always uppercase)
timestamp	timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback
local_timestamp	message arrival timestamp in microseconds since epoch
id	liquidation id as provided by exchange, empty if exchange does not provide one - different exchanges provide id's as numeric values, GUID's or other strings, and some do not provide that information at all
side	liquidation side: buy - short position was liquidated sell - long position was liquidated
price	liquidation price as provided by exchange
amount	liquidation amount as provided by exchange

dataset preview

exchange	symbol	timestamp	local_timestamp	id	side	price	amount
binance-futures	BTCUSDT	1632009737493000	1632009737505152		sell	48283.81	0.01
binance-futures	BTCUSDT	1632009802385000	1632009802398690		buy	48339.11	0.132
binance-futures	BTCUSDT	1632009870475000	1632009870485139		buy	48337.02	0.004
binance-futures	BTCUSDT	1632009889760000	1632009889784144		buy	48346.54	0.002
binance-futures	BTCUSDT	1632009891282000	1632009891296156		buy	48350.34	0.032
binance-futures	BTCUSDT	1632009892636000	1632009892646433		buy	48355.68	0.001
binance-futures	BTCUSDT	1632009970533000	1632009970544039		buy	48290.57	0.042
binance-futures	BTCUSDT	1632010836285000	1632010836297995		sell	48186.15	0.036
binance-futures	BTCUSDT	1632010899415000	1632010899428203		sell	48150.64	0.265

https://datasets.tardis.dev/v1/ftx/liquidations/2021/09/01/PERPETUALS.csv.gz

FTX perpetual futures liquidations for 2021-09-01

https://datasets.tardis.dev/v1/bitmex/liquidations/2021/09/01/XBTUSD.csv.gz

BitMEX XBTUSD liquidations for 2021-09-91

Grouped symbols

In addition to standard currency pairs & instrument symbols that can be requested when via CSV datasets API, each exchange has additional special grouped symbols available depending if it supports given market type: SPOT, FUTURES, OPTIONS and PERPETUALS. When such symbol is requested then downloaded file for it has all the data for all instruments belonging for given market type. This is especially useful for options instruments that as specifying each option symbol one by one can be mundane process, using 'OPTIONS' as a symbol gives data for all options available at given time.

See what is the difference between futures and perpetual swaps contracts.

• incremental_book_L2 - available for FUTURES

  • Examples:

    • https://datasets.tardis.dev/v1/ftx/incremental_book_L2/2020/09/01/FUTURES.csv.gz

• trades - available for SPOT, FUTURES, OPTIONS, PERPETUALS

  • Examples:

    • https://datasets.tardis.dev/v1/binance/trades/2020/09/01/SPOT.csv.gz

    • https://datasets.tardis.dev/v1/deribit/trades/2020/09/01/FUTURES.csv.gz

    • https://datasets.tardis.dev/v1/okex-options/trades/2020/09/01/OPTIONS.csv.gz

    • https://datasets.tardis.dev/v1/ftx/trades/2020/09/01/PERPETUALS.csv.gz

• derivative_ticker - available for FUTURES, PERPETUALS

  • Examples:

    • https://datasets.tardis.dev/v1/ftx/derivative_ticker/2020/09/01/FUTURES.csv.gz

    • https://datasets.tardis.dev/v1/bitmex/derivative_ticker/2020/09/01/PERPETUALS.csv.gz

• options_chain - available for OPTIONS

  • Examples:

    • https://datasets.tardis.dev/v1/deribit/options_chain/2020/09/01/OPTIONS.csv.gz

    • https://datasets.tardis.dev/v1/okex-options/options_chain/2020/09/01/OPTIONS.csv.gz

• quotes - available for OPTIONS

  • Examples:

    • https://datasets.tardis.dev/v1/deribit/quotes/2020/09/01/OPTIONS.csv.gz

    • https://datasets.tardis.dev/v1/okex-options/quotes/2020/09/01/OPTIONS.csv.gz

• liquidations - available for FUTURES, PERPETUALS, OPTIONS

  • Examples:

    • https://datasets.tardis.dev/v1/deribit/liquidations/2021/09/01/PERPETUALS.csv.gz

    • https://datasets.tardis.dev/v1/ftx/liquidations/2021/09/01/FUTURES.csv.gz

    • https://datasets.tardis.dev/v1/deribit/liquidations/2021/09/01/OPTIONS.csv.gz

those special symbols are also listed in response to /exchanges/:exchange API call

Datasets API details

• all downloadable datasets are gzip compressed

• historical market data is available in daily intervals (separate file for each day) based on local timestamp (timestamp of message arrival) split by exchange, data type and symbol

• data for a given day is available on the next day around 6h after 00:00 UTC - exact date until when data is available can be requested via /exchanges/:exchange API call (datasets.exportedUntil), e.g., https://api.tardis.dev/v1/exchanges/ftx

• datasets are ordered and split into separate daily files by local_timestamp (timestamp of message arrival time)

• empty gzip compressed file is being returned in case of no data available for a given day, symbol and data type, e.g., exchange downtime, very low volume currency pairs etc.

• iftimestamp equals to local_timestamp it means that exchange didn't provide timestamp for message, e.g., BitMEX order book updates

• cell in CSV file is empty if there's no value for it, e.g., no trade id if a given exchange doesn't provide one

• datasets are sourced from Tardis.dev HTTP API, which in turn provides the the data sourced from exchanges real-time WebSocket market data feeds (in contrast to REST API endpoints)

• See "Data FAQ" regarding potential order book overlaps issues, non monotonically increasing exchanges timestamps, duplicated trade data and more

Download via client libraries

Historical datasets for the first day of each month are available to download without API key.

Python

# pip install tardis-dev
# requires Python >=3.6
from tardis_dev import datasets, get_exchange_details
import logging

# comment out to disable debug logs
logging.basicConfig(level=logging.DEBUG)

# function used by default if not provided via options
def default_file_name(exchange, data_type, date, symbol, format):
    return f"{exchange}_{data_type}_{date.strftime('%Y-%m-%d')}_{symbol}.{format}.gz"


# customized get filename function - saves data in nested directory structure
def file_name_nested(exchange, data_type, date, symbol, format):
    return f"{exchange}/{data_type}/{date.strftime('%Y-%m-%d')}_{symbol}.{format}.gz"


# returns data available at https://api.tardis.dev/v1/exchanges/deribit
deribit_details = get_exchange_details("deribit")
# print(deribit_details)

datasets.download(
    # one of https://api.tardis.dev/v1/exchanges with supportsDatasets:true - use 'id' value
    exchange="deribit",
    # accepted data types - 'datasets.symbols[].dataTypes' field in https://api.tardis.dev/v1/exchanges/deribit,
    # or get those values from 'deribit_details["datasets"]["symbols][]["dataTypes"] dict above
    data_types=["incremental_book_L2", "trades", "quotes", "derivative_ticker", "book_snapshot_25", "book_snapshot_5", "liquidations"],
    # change date ranges as needed to fetch full month or year for example
    from_date="2019-11-01",
    # to date is non inclusive
    to_date="2019-11-02",
    # accepted values: 'datasets.symbols[].id' field in https://api.tardis.dev/v1/exchanges/deribit
    symbols=["BTC-PERPETUAL", "ETH-PERPETUAL",],
    # (optional) your API key to get access to non sample data as well
    api_key="YOUR API KEY",
    # (optional) path where data will be downloaded into, default dir is './datasets'
    # download_dir="./datasets",
    # (optional) - one can customize downloaded file name/path (flat dir strucure, or nested etc) - by default function 'default_file_name' is used
    # get_filename=default_file_name,
    # (optional) file_name_nested will download data to nested directory structure (split by exchange and data type)
    # get_filename=file_name_nested,
)

If you're running into RuntimeError: This event loop is already running error try solution from https://github.com/ipython/ipython/issues/11338#issuecomment-646539516 (adding nest_asyncio).

Node.js

// npm install [email protected]
// requires node version >=12

// remove it to disable debug logs
process.env.DEBUG = 'tardis-dev*'

const { downloadDatasets, getExchangeDetails } = require('tardis-dev')

;(async () => {
  // returns data available at https://api.tardis.dev/v1/exchanges/deribit
  const deribitDetails = await getExchangeDetails('deribit')

  // console.log(deribitDetails.datasets)

  await downloadDatasets({
    exchange: 'deribit', // one of https://api.tardis.dev/v1/exchanges with supportsDatasets:true - use 'id' value
    dataTypes: ['incremental_book_L2', 'trades', 'quotes', 'derivative_ticker', 'book_snapshot_25', 'book_snapshot_5', 'liquidations'], // accepted data types - 'datasets.symbols[].dataTypes' field in https://api.tardis.dev/v1/exchanges/deribit, or get those values from 'deribitDetails.datasets.symbols[].dataTypes' object above
    from: '2019-11-01', // change date ranges as needed to fetch full month or year for example
    to: '2019-11-02', // to date is non inclusive
    symbols: ['BTC-PERPETUAL', 'ETH-PERPETUAL'], // accepted values: 'datasets.symbols[].id' field in https://api.tardis.dev/v1/exchanges/deribit, or `deribitDetails.datasets.symbols[].id` from object above

    apiKey: 'YOUR_API_KEY', // (optional) your API key to get access to non sample data as well
    // downloadDir:'./datasets', // (optional) path where data will be downloaded into, default dir is './datasets'

    // getFilename: getFilenameDefault, // (optional) - one can customize downloaded file name/path (flat dir strucure, or nested etc) - by default function 'getFilenameDefault' is used
    // getFilename: getFilenameCustom // (optional) getFilenameCustom will download data to nested directory structure (split by exchange and data type)
  })
})().catch((e) => {
  console.log('download error', e)
})

// function used by default if not provided via options
function getFilenameDefault({ exchange, dataType, format, date, symbol }) {
  return `${exchange}_${dataType}_${date.toISOString().split('T')[0]}_${symbol}.${format}.gz`
}

// customized get filename function - saves data in nested directory structure
function getFilenameCustom({ exchange, dataType, format, date, symbol }) {
  return `${exchange}/${dataType}/${date.toISOString().split('T')[0]}_${symbol}.${format}.gz`
}

Datasets API reference

GET https://datasets.tardis.dev/v1/:exchange/:dataType/:year/:month/:day/:symbol.csv.gz

Returns gzip compressed CSV dataset for given exchange, data type, date (year, month, day) and symbol.

Path Parameters

Name	Type	Description
exchange	string	one of https://api.tardis.dev/v1/exchanges (field id, only exchanges with "supportsDatasets":true)
dataType	string	one of datasets.symbols[].dataTypes values from https://api.tardis.dev/v1/exchanges/:exchange API response
year	string	year in format YYYY (four-digit year)
month	string	month in format MM (two-digit month of the year)
day	string	day in format DD (two-digit day of the month)
symbol	string	one of datasets.symbols[].id values from https://api.tardis.dev/v1/exchanges/:exchange API response, see details below

Headers

Name	Type	Description
Authorization	string	For authenticated requests provide Authorization header with value: 'Bearer YOUR_API_KEY'. Without API key historical datasets for the first day of each month are available to download.

200 gzip compressed CSV dataset

• symbols param provided to datasets API in comparison to HTTP API needs to be both always uppercase and have '/' and ':' characters replaced with '-' so symbol is url safe.

• list of allowed symbols for each exchange can be requested via /exchanges/:exchange API call, e.g., https://api.tardis.dev/v1/exchanges/deribit - datasets.symbols[].id field

Sample requests

https://datasets.tardis.dev/v1/deribit/trades/2019/11/01/BTC-PERPETUAL.csv.gz

https://datasets.tardis.dev/v1/deribit/trades/2019/11/01/OPTIONS.csv.gz

https://datasets.tardis.dev/v1/bitmex/incremental_book_L2/2020/04/01/XBTUSD.csv.gz

https://datasets.tardis.dev/v1/deribit/options_chain/2019/08/01/OPTIONS.csv.gz

https://datasets.tardis.dev/v1/deribit/book_snapshot_25/2020/08/01/BTC-PERPETUAL.csv.gz

https://datasets.tardis.dev/v1/deribit/quotes/2019/08/01/OPTIONS.csv.gz