Downloadable CSV files
CSV datasets are available via dedicated datasets API that allows downloading tick level incremental order book L2 updates, order book snapshots, trades, options chains, quotes, derivative tickers and liquidations data.
For ongoing data, CSV datasets for a given day are available on the next day around 05:00 UTC.
CSV datasets are exported from exchanges' real-time WebSocket feeds data we collected (and also provide via our API as historical data in exchange-native format).
Historical datasets for the first day of each month are available to download without API key.
Our Node.js and Python clients have built-in functions to efficiently download whole date range of data.
Python
Node.js
cURL
# pip install tardis-dev
# requires Python >=3.6
from tardis_dev import datasets
datasets.download(
exchange="deribit",
data_types=[
"incremental_book_L2",
"trades",
"quotes",
"derivative_ticker",
"book_snapshot_25",
"liquidations"
],
from_date="2019-11-01",
to_date="2019-11-02",
symbols=["BTC-PERPETUAL", "ETH-PERPETUAL"],
api_key="YOUR API KEY (optionally)",
)
See full example that shows all available download options (download path customization, filenames conventions and more).
// npm install [email protected]
// requires node version >=12
const { downloadDatasets } = require('tardis-dev')
;(async () => {
await downloadDatasets({
exchange: 'deribit',
dataTypes: [
'incremental_book_L2',
'trades',
'quotes',
'derivative_ticker',
'book_snapshot_25',
'liquidations'
],
from: '2019-11-01',
to: '2019-11-02',
symbols: ['BTC-PERPETUAL', 'ETH-PERPETUAL'],
apiKey: 'YOUR API KEY (optionally)'
})
})()
See full example that shows all available download options (download path customization, filenames conventions and more).
curl -o deribit_trades_2019-11-01_BTC-PERPETUAL.csv.gz https://datasets.tardis.dev/v1/deribit/trades/2019/11/01/BTC-PERPETUAL.csv.gz
- columns delimiter: , (comma)
- new line marker: \n (LF)
- decimal mark: . (dot)
- date time timezone: UTC
Incremental order book L2 updates collected from exchanges' real-time WebSocket order book L2 data feeds - data as deep and granular as underlying real-time data source, please see FAQ: What is the maximum order book depth available for each supported exchange? for more details.
As exchanges real-time feeds usually publish multiple order book levels updates via single message you can recognize that by grouping rows by
local_timestamp
field if needed. If you have any doubts how to correctly reconstruct full order book state from incremental_book_L2 CSV dataset, please see this answer or contact us.
In case you only need order book data for top 25 or top 5 levels, we do provide datasets with already reconstructed snapshots for every update for those. See book_snapshot_25 and book_snapshot_5.
CSV incremental_book_L2 schema
dataset preview
column name | description | |
exchange | | |
symbol | instrument symbol as provided by exchange (always uppercase) | |
timestamp | timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback | |
local_timestamp | message arrival timestamp in microseconds since epoch | |
is_snapshot | possible values:
If last update was not a snapshot and current one is, then existing order book state must be discarded (all existing levels removed) | |
side | determines to which side of the order book update belongs to:
| |
price | price identifying book level being updated | |
amount | updated price level amount as provided by exchange, not a delta - an amount of 0 indicates that the price level can be removed | |
exchange | symbol | timestamp | local_timestamp | is_snapshot | side | price | amount |
deribit | BTC-PERPETUAL | 1585699209920000 | 1585699209934201 | false | ask | 6443.5 | 38640 |
deribit | BTC-PERPETUAL | 1585699209947000 | 1585699209957629 | false | bid | 6311.5 | 0 |
deribit | BTC-PERPETUAL | 1585699209950000 | 1585699209963464 | false | ask | 6428 | 13210 |
deribit | BTC-PERPETUAL | 1585699209967000 | 1585699209979152 | false | bid | 6311.5 | 750 |
deribit | BTC-PERPETUAL | 1585699209970000 | 1585699209983585 | false | bid | 6327 | 16010 |
deribit | BTC-PERPETUAL | 1585699209970000 | 1585699209983585 | false | bid | 6325 | 210530 |
deribit | BTC-PERPETUAL | 1585699209972000 | 1585699209983691 | false | bid | 6351 | 810 |
deribit | BTC-PERPETUAL | 1585699209972000 | 1585699209983691 | false | bid | 6352.5 | 18830 |
deribit | BTC-PERPETUAL | 1585699209974000 | 1585699209983703 | false | ask | 6492 | 100 |
https://datasets.tardis.dev/v1/deribit/incremental_book_L2/2020/04/01/BTC-PERPETUAL.csv.gz
Deribit BTC-PERPETUAL incremental order book L2 updates for 2020-04-01
https://datasets.tardis.dev/v1/deribit/incremental_book_L2/2020/09/01/FUTURES.csv.gz
Deribit FUTURES instruments incremental order book L2 updates for 2020-09-01
Tick-level order book snapshots reconstructed from exchanges' real-time WebSocket order book L2 data feeds. Each row represents top 25 levels from each side of the limit order book book and was recorded every time any of the tracked bids/asks top 25 levels have changed.
CSV book_snapshot_25 schema
dataset preview
column name | description | |
exchange | | |
symbol | instrument symbol as provided by exchange (always uppercase) | |
timestamp | timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback | |
local_timestamp | message arrival timestamp in microseconds since epoch | |
asks[0..24].price | top 25 asks prices in ascending order, empty if there aren't enough price levels available in the order book or provided by the exchange | |
asks[0..24].amount | top 25 asks amounts in ascending order, empty if there aren't enough price levels available in the order book or provided by the exchange | |
bids[0..24].price | top 25 bids prices in descending order, empty if there aren't enough price levels available in the order book or provided by the exchange | |
bids[0..24].amount | top 25 bids amounts in descending order, empty if there aren't enough price levels available in the order book or provided by the exchange | |
exchange | symbol | timestamp | local_timestamp | asks[0].price | asks[0].amount | bids[0].price | bids[0].amount | asks[1].price | asks[1].amount | bids[1].price | bids[1].amount | asks[2].price | asks[2].amount | bids[2].price | bids[2].amount | asks[3].price | asks[3].amount | bids[3].price | bids[3].amount | asks[4].price | asks[4].amount | bids[4].price | bids[4].amount | asks[5].price | asks[5].amount | bids[5].price | bids[5].amount | asks[6].price | asks[6].amount | bids[6].price | bids[6].amount | asks[7].price | asks[7].amount | bids[7].price | bids[7].amount | asks[8].price | asks[8].amount | bids[8].price | bids[8].amount | asks[9].price | asks[9].amount | bids[9].price | bids[9].amount | asks[10].price | asks[10].amount | bids[10].price | bids[10].amount | asks[11].price | asks[11].amount | bids[11].price | bids[11].amount | asks[12].price | asks[12].amount | bids[12].price | bids[12].amount | asks[13].price | asks[13].amount | bids[13].price | bids[13].amount | asks[14].price | asks[14].amount | bids[14].price | bids[14].amount | asks[15].price | asks[15].amount | bids[15].price | bids[15].amount | asks[16].price | asks[16].amount | bids[16].price | bids[16].amount | asks[17].price | asks[17].amount | bids[17].price | bids[17].amount | asks[18].price | asks[18].amount | bids[18].price | bids[18].amount | asks[19].price | asks[19].amount | bids[19].price | bids[19].amount | asks[20].price | asks[20].amount | bids[20].price | bids[20].amount | asks[21].price | asks[21].amount | bids[21].price | bids[21].amount | asks[22].price | asks[22].amount | bids[22].price | bids[22].amount | asks[23].price | asks[23].amount | bids[23].price | bids[23].amount | asks[24].price | asks[24].amount | bids[24].price | bids[24].amount |
deribit | BTC-PERPETUAL | 1599868800206000 | 1599868800253274 | 10396 | 48050 | 10395.5 | 18220 | 10396.5 | 22220 | 10395 | 16570 | 10397 | 100 | 10394.5 | 22630 | 10397.5 | 8360 | 10394 | 16670 | 10398 | 1500 | 10393.5 | 16570 | 10398.5 | 13210 | 10393 | 5600 | 10399.5 | 60070 | 10392.5 | 20500 | 10400 | 5100 | 10392 | 30 | 10400.5 | 5140 | 10391.5 | 75780 | 10401 | 13040 | 10391 | 12110 | 10401.5 | 2250 | 10390.5 | 280 | 10402 | 9150 | 10390 | 52680 | 10402.5 | 119390 | 10389.5 | 18240 | 10403 | 23070 | 10389 | 73010 | 10403.5 | 53930 | 10388.5 | 67500 | 10404 | 43590 | 10388 | 313140 | 10404.5 | 271050 | 10387.5 | 280 | 10405 | 73710 | 10387 | 9840 | 10405.5 | 32480 | 10386.5 | 104570 | 10406 | 41220 | 10386 | 269050 | 10406.5 | 20400 | 10385.5 | 21840 | 10407 | 45460 | 10385 | 79000 | 10407.5 | 69630 | 10384.5 | 220 | 10408 | 22230 | 10384 | 71440 | 10408.5 | 30840 | 10383.5 | 44740 |
deribit | BTC-PERPETUAL | 1599868800280000 | 1599868800310441 | 10396 | 48050 | 10395.5 | 18220 | 10396.5 | 22220 | 10395 | 16570 | 10397 | 100 | 10394.5 | 22630 | 10397.5 | 8360 | 10394 | 16670 | 10398 | 1500 | 10393.5 | 16570 | 10398.5 | 13210 | 10393 | 5600 | 10399.5 | 60070 | 10392.5 | 20500 | 10400 | 5100 | 10392 | 30 | 10400.5 | 5140 | 10391.5 | 75780 | 10401 | 13040 | 10391 | 12110 | 10401.5 | 2250 | 10390.5 | 280 | 10402 | 9150 | 10390 | 52680 | 10402.5 | 119390 | 10389.5 | 18240 | 10403 | 23070 | 10389 | 73010 | 10403.5 | 53930 | 10388.5 | 67500 | 10404 | 43590 | 10388 | 313140 | 10404.5 | 271050 | 10387.5 | 280 | 10405 | 73710 | 10387 | 9850 | 10405.5 | 32480 | 10386.5 | 104570 | 10406 | 41220 | 10386 | 269050 | 10406.5 | 20400 | 10385.5 | 21840 | 10407 | 45460 | 10385 | 79000 | 10407.5 | 69630 | 10384.5 | 220 | 10408 | 22230 | 10384 | 71440 | 10408.5 | 30840 | 10383.5 | 44740 |
deribit | BTC-PERPETUAL | 1599868814801000 | 1599868814817631 | 10398.5 | 20 | 10398 | 7400 | 10399 | 4890 | 10397.5 | 17680 | 10399.5 | 520 | 10396.5 | 17680 | 10400 | 1700 | 10396 | 30280 | 10400.5 | 3010 | 10395.5 | 44110 | 10401 | 40 | 10395 | 20080 | 10401.5 | 2570 | 10394.5 | 91410 | 10402 | 400 | 10394 | 97570 | 10402.5 | 50530 | 10393.5 | 27510 | 10403 | 9960 | 10393 | 3330 | 10403.5 | 54250 | 10392.5 | 200 | 10404 | 40 | 10392 | 20400 | 10404.5 | 10 | 10391.5 | 75650 | 10405 | 93470 | 10391 | 9580 | 10405.5 | 32540 | 10390.5 | 260040 | 10406 | 26130 | 10390 | 310 | 10406.5 | 9670 | 10389.5 | 21210 | 10407 | 1180 | 10389 | 87320 | 10407.5 | 89030 | 10388.5 | 61140 | 10408 | 54860 | 10388 | 283120 | 10408.5 | 42430 | 10387.5 | 10680 | 10409 | 260680 | 10387 | 11400 | 10409.5 | 19220 | 10386.5 | 92470 | 10410 | 94970 | 10386 | 49640 | 10410.5 | 50 | 10385.5 | 6420 |
deribit | BTC-PERPETUAL | 1599868814809000 | 1599868814817632 | 10398.5 | 20 | 10398 | 7400 | 10399 | 4890 | 10397.5 | 17680 | 10399.5 | 520 | 10396.5 | 17680 | 10400 | 1700 | 10396 | 30280 | 10400.5 | 3010 | 10395.5 | 44110 | 10401 | 40 | 10395 | 20080 | 10401.5 | 2570 | 10394.5 | 91410 | 10402 | 400 | 10394 | 97570 | 10402.5 | 50530 | 10393.5 | 27510 | 10403 | 9960 | 10393 | 3330 | 10403.5 | 54900 | 10392.5 | 200 | 10404 | 40 | 10392 | 20400 | 10404.5 | 10 | 10391.5 | 75650 | 10405 | 93470 | 10391 | 9580 | 10405.5 | 32540 | 10390.5 | 260040 | 10406 | 26130 | 10390 | 310 | 10406.5 | 9670 | 10389.5 | 21210 | 10407 | 1180 | 10389 | 87320 | 10407.5 | 89030 | 10388.5 | 61140 | 10408 | 54860 | 10388 | 283120 | 10408.5 | 42430 | 10387.5 | 10680 | 10409 | 260680 | 10387 | 11400 | 10409.5 | 19220 | 10386.5 | 92470 | 10410 | 94970 | 10386 | 49640 | 10410.5 | 50 | 10385.5 | 6420 |
deribit | BTC-PERPETUAL | 1599868815411000 | 1599868815414125 | 10399 | 4910 | 10398 | 25080 | 10399.5 | 20 | 10397.5 | 17680 | 10400 | 2200 | 10396.5 | 17680 | 10400.5 | 2910 | 10396 | 31780 | 10401 | 40 | 10395.5 | 44110 | 10401.5 | 570 | 10395 | 20050 | 10402 | 500 | 10394.5 | 91440 | 10402.5 | 52990 | 10394 | 98510 | 10403 | 3500 | 10393.5 | 26570 | 10403.5 | 45100 | 10393 | 3330 | 10404 | 9190 | 10392.5 | 470 | 10404.5 | 10 | 10392 | 18300 | 10405 | 70030 | 10391.5 | 85130 | 10405.5 | 60800 | 10391 | 8640 | 10406 | 26130 | 10390.5 | 260040 | 10406.5 | 9270 | 10390 | 22530 | 10407 | 240 | 10389.5 | 14030 | 10407.5 | 89970 | 10389 | 65120 | 10408 | 23640 | 10388.5 | 72380 | 10408.5 | 62090 | 10388 | 283120 | 10409 | 260680 | 10387.5 | 10280 | 10409.5 | 18150 | 10387 | 11400 | 10410 | 94970 | 10386.5 | 123630 | 10410.5 | 50 | 10386 | 8470 | 10411 | 28210 | 10385.5 | 6420 |
deribit | BTC-PERPETUAL | 1599868815411000 | 1599868815419035 | 10399 | 4910 | 10398 | 25080 | 10399.5 | 20 | 10397.5 | 17680 | 10400 | 2200 | 10396.5 | 17680 | 10400.5 | 2910 | 10396 | 31780 | 10401 | 40 | 10395.5 | 44110 | 10401.5 | 570 | 10395 | 20050 | 10402 | 500 | 10394.5 | 91440 | 10402.5 | 52990 | 10394 | 98510 | 10403 | 3500 | 10393.5 | 26570 | 10403.5 | 45100 | 10393 | 3330 | 10404 | 9190 | 10392.5 | 470 | 10404.5 | 10 | 10392 | 18300 | 10405 | 70030 | 10391.5 | 85130 | 10405.5 | 60800 | 10391 | 8640 | 10406 | 26130 | 10390.5 | 260040 | 10406.5 | 17270 | 10390 | 22530 | 10407 | 240 | 10389.5 | 14030 | 10407.5 | 89970 | 10389 | 65120 | 10408 | 23640 | 10388.5 | 72380 | 10408.5 | 62090 | 10388 | 283120 | 10409 | 260680 | 10387.5 | 10280 | 10409.5 | 18150 | 10387 | 11400 | 10410 | 94970 | 10386.5 | 123630 | 10410.5 | 50 | 10386 | 8470 | 10411 | 28210 | 10385.5 | 6420 |
deribit | BTC-PERPETUAL | 1599868907943000 | 1599868907946933 | 10398 | 4600 | 10397.5 | 73090 | 10399.5 | 10 | 10397 | 24630 | 10400 | 3300 | 10396.5 | 22770 | 10400.5 | 10270 | 10396 | 3130 | 10401 | 25390 | 10395.5 | 5000 | 10401.5 | 119790 | 10395 | 9060 | 10402 | 8510 | 10394.5 | 17910 | 10402.5 | 8180 | 10394 | 138990 | 10403 | 10000 | 10393.5 | 33080 | 10404 | 7960 | 10393 | 3020 | 10404.5 | 15130 | 10392.5 | 8130 | 10405 | 128930 | 10392 | 100920 | 10405.5 | 109560 | 10391.5 | 83330 | 10406 | 8610 | 10391 | 32220 | 10406.5 | 34890 | 10390.5 | 278270 | 10407 | 44440 | 10390 | 47980 | 10407.5 | 102620 | 10389.5 | 292240 | 10408 | 20660 | 10389 | 65100 | 10408.5 | 175160 | 10388.5 | 790 | 10409 | 7660 | 10388 | 55720 | 10409.5 | 308550 | 10387.5 | 31440 | 10410 | 138130 | 10387 | 3830 | 10410.5 | 15940 | 10386.5 | 109470 | 10411 | 2610 | 10386 | 31560 | 10411.5 | 3780 | 10385.5 | 3450 |
deribit | BTC-PERPETUAL | 1599868907944000 | 1599868907953129 | 10398 | 4600 | 10397.5 | 73090 | 10399.5 | 10 | 10397 | 24630 | 10400 | 3300 | 10396.5 | 22770 | 10400.5 | 10270 | 10396 | 3130 | 10401 | 25390 | 10395.5 | 5000 | 10401.5 | 119790 | 10395 | 1060 | 10402 | 8510 | 10394.5 | 17910 | 10402.5 | 8180 | 10394 | 146990 | 10403 | 10000 | 10393.5 | 33080 | 10404 | 7960 | 10393 | 3020 | 10404.5 | 15130 | 10392.5 | 8130 | 10405 | 128930 | 10392 | 100920 | 10405.5 | 109560 | 10391.5 | 83330 | 10406 | 8610 | 10391 | 32220 | 10406.5 | 34890 | 10390.5 | 278270 | 10407 | 44440 | 10390 | 47980 | 10407.5 | 102620 | 10389.5 | 292240 | 10408 | 20660 | 10389 | 65100 | 10408.5 | 175160 | 10388.5 | 790 | 10409 | 7660 | 10388 | 55720 | 10409.5 | 308550 | 10387.5 | 31440 | 10410 | 138130 | 10387 | 3830 | 10410.5 | 15940 | 10386.5 | 109470 | 10411 | 2610 | 10386 | 31560 | 10411.5 | 3780 | 10385.5 | 3450 |
deribit | BTC-PERPETUAL | 1599868907993000 | 1599868907997022 | 10398 | 4600 | 10397.5 | 73090 | 10399.5 | 2010 | 10397 | 24630 | 10400 | 3300 | 10396.5 | 22770 | 10400.5 | 8270 | 10396 | 3130 | 10401 | 25390 | 10395.5 | 5000 | 10401.5 | 119790 | 10395 | 1060 | 10402 | 8510 | 10394.5 | 17910 | 10402.5 | 8180 | 10394 | 146990 | 10403 | 10000 | 10393.5 | 33080 | 10404 | 7960 | 10393 | 3020 | 10404.5 | 15130 | 10392.5 | 8130 | 10405 | 128930 | 10392 | 100920 | 10405.5 | 109560 | 10391.5 | 83330 | 10406 | 8610 | 10391 | 32220 | 10406.5 | 34890 | 10390.5 | 278270 | 10407 | 44440 | 10390 | 47980 | 10407.5 | 102620 | 10389.5 | 292240 | 10408 | 20660 | 10389 | 65100 | 10408.5 | 175160 | 10388.5 | 790 | 10409 | 7660 | 10388 | 55720 | 10409.5 | 308550 | 10387.5 | 31440 | 10410 | 138130 | 10387 | 3830 | 10410.5 | 15940 | 10386.5 | 109470 | 10411 | 2610 | 10386 | 31560 | 10411.5 | 3780 | 10385.5 | 3450 |
https://datasets.tardis.dev/v1/bitmex/book_snapshot_25/2020/09/01/XBTUSD.csv.gz
BitMEX XBTUSD top 25 levels order book snapshots for 2020-09-01
https://datasets.tardis.dev/v1/binance-futures/book_snapshot_25/2020/09/01/BTCUSDT.csv.gz
Binance USDT Futures BTCUSDT top 25 levels order book snapshots for 2020-09-01
Tick-level order book snapshots reconstructed from exchanges' real-time WebSocket order book L2 data feeds. Each row represents top 5 levels from each side of the limit order book book and was recorded every time any of the tracked bids/asks top 5 levels have changed.
CSV book_snapshot_5 schema
dataset preview
column name | description | |
exchange | | |
symbol | instrument symbol as provided by exchange (always uppercase) | |
timestamp | timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback | |
local_timestamp | message arrival timestamp in microseconds since epoch | |
asks[0..4].price | top 5 asks prices in ascending order, empty if there aren't enough price levels available in the order book or provided by the exchange | |
asks[0..4].amount | top 5 asks amounts in ascending order, empty if there aren't enough price levels available in the order book or provided by the exchange | |
bids[0..4].price | top 5 bids prices in descending order, empty if there aren't enough price levels available in the order book or provided by the exchange | |
bids[0..4].amount | top 5 bids amounts in descending order, empty if there aren't enough price levels available in the order book or provided by the exchange | |
exchange | symbol | timestamp | local_timestamp | asks[0].price | asks[0].amount | bids[0].price | bids[0].amount | asks[1].price | asks[1].amount | bids[1].price | bids[1].amount | asks[2].price | asks[2].amount | bids[2].price | bids[2].amount | asks[3].price | asks[3].amount | bids[3].price | bids[3].amount | asks[4].price | asks[4].amount |
bitmex | XBTUSD | 1598918402683390 | 1598918402683390 | 11658 | 1399982 | 11657.5 | 2293327 | 11658.5 | 82328 | 11657 | 37555 | 11659 | 3001 | 11656.5 | 110647 | 11659.5 | 10843 | 11656 | 10063 | 11660 | 2522 |
bitmex | XBTUSD | 1598918403229829 | 1598918403229829 | 11658 | 1399982 | 11657.5 | 2293327 | 11658.5 | 82328 | 11657 | 37555 | 11659 | 3001 | 11656.5 | 110647 | 11659.5 | 10835 | 11656 | 10063 | 11660 | 2522 |
bitmex | XBTUSD | 1598918403232925 | 1598918403232925 | 11658 | 1399982 | 11657.5 | 2295327 | 11658.5 | 82328 | 11657 | 37555 | 11659 | 3001 | 11656.5 | 110647 | 11659.5 | 10835 | 11656 | 10063 | 11660 | 2522 |
bitmex | XBTUSD | 1598918403253585 | 1598918403253585 | 11658 | 1399982 | 11657.5 | 2295256 | 11658.5 | 82328 | 11657 | 37555 | 11659 | 3001 | 11656.5 | 110647 | 11659.5 | 10835 | 11656 | 10063 | 11660 | 2522 |
bitmex | XBTUSD | 1598918403256460 | 1598918403256460 | 11658 | 1399982 | 11657.5 | 2294159 | 11658.5 | 82328 | 11657 | 37555 | 11659 | 3001 | 11656.5 | 110647 | 11659.5 | 10835 | 11656 | 10063 | 11660 | 2522 |
bitmex | XBTUSD | 1598947264502293 | 1598947264502293 | 11950 | 730542 | 11949.5 | 1100309 | 11950.5 | 30454 | 11949 | 454098 | 11951 | 72967 | 11948.5 | 2519605 | 11951.5 | 48967 | 11948 | 97449 | 11952 | 55623 |
bitmex | XBTUSD | 1598947264505452 | 1598947264505452 | 11950 | 730542 | 11949.5 | 1100309 | 11950.5 | 30454 | 11949 | 454098 | 11951 | 72967 | 11948.5 | 2509605 | 11951.5 | 48967 | 11948 | 97449 | 11952 | 55623 |
bitmex | XBTUSD | 1598947264510015 | 1598947264510015 | 11950 | 730542 | 11949.5 | 1100975 | 11950.5 | 30454 | 11949 | 454098 | 11951 | 72967 | 11948.5 | 2509605 | 11951.5 | 48967 | 11948 | 97449 | 11952 | 55623 |
bitmex | XBTUSD | 1598947264510024 | 1598947264510024 | 11950 | 730542 | 11949.5 | 1250975 | 11950.5 | 30454 | 11949 | 454098 | 11951 | 72967 | 11948.5 | 2509605 | 11951.5 | 48967 | 11948 | 97449 | 11952 | 55623 |
https://datasets.tardis.dev/v1/bitmex/book_snapshot_5/2020/09/01/XBTUSD.csv.gz
BitMEX XBTUSD top 5 levels order book snapshots for 2020-09-01
https://datasets.tardis.dev/v1/binance-futures/book_snapshot_5/2020/09/01/BTCUSDT.csv.gz
Binance USDT Futures BTCUSDT top 5 levels order book snapshots for 2020-09-01
CSV trades schema
dataset preview
column name | description | |
exchange | | |
symbol | instrument symbol as provided by exchange (always uppercase) | |
timestamp | timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback | |
local_timestamp | message arrival timestamp in microseconds since epoch | |
id | trade id as provided by exchange, empty if exchange does not provide one - different exchanges provide id's as numeric values, GUID's or other strings, and some do not provide that information at all | |
side | liquidity taker side (aggressor), possible values:
| |
price | trade price as provided by exchange | |
amount | trade amount as provided by exchange | |
exchange | symbol | timestamp | local_timestamp | id | side | price | amount |
bitmex | XBTUSD | 1585699202957000 | 1585699203089980 | d20... | buy | 6425.5 | 12 |
bitmex | XBTUSD | 1585699202980000 | 1585699203095276 | 619... | sell | 6425 | 150 |
bitmex | XBTUSD | 1585699203002000 | 1585699203099299 | 751... | sell | 6425 | 25 |
bitmex | XBTUSD | 1585699203092000 | 1585699203122233 | 3c1... | buy | 6425.5 | 1 |
bitmex | XBTUSD | 1585699203092000 | 1585699203122233 | b9b... | buy | 6425.5 | 1 |
bitmex | XBTUSD | 1585699203092000 | 1585699203122233 | 433... | buy | 6425.5 | 1 |
bitmex | XBTUSD | 1585699203092000 | 1585699203122233 | d16... | buy | 6425.5 | 1 |
bitmex | XBTUSD | 1585699203092000 | 1585699203122233 | 402... | buy | 6425.5 | 1 |
bitmex | XBTUSD | 1585699203092000 | 1585699203122233 | 2f8... | buy | 6425.5 | 1 |
https://datasets.tardis.dev/v1/bitmex/trades/2020/03/01/XBTUSD.csv.gz
Bitmex XBTUSD trades for 2020-03-01 dataset sample
https://datasets.tardis.dev/v1/okex-futures/trades/2020/03/01/FUTURES.csv.gz
OKEx Futures FUTURES instruments trades for 2020-03-01 dataset sample
Tick-level options summary info (strike prices, expiration dates, open interest, implied volatility, greeks etc.) for all active options instruments collected from exchanges' real-time WebSocket options tickers data feeds. Options chain data is available for Deribit (sourced from ticker channel) and OKEx Options (sourced from option/summary and index/ticker channels).
For
options_chain
data type only 'OPTIONS' symbol is available (one file per day for all options instruments).
CSV options_chain schema
dataset preview
column name | description | |
exchange | | |
symbol | instrument symbol as provided by exchange (always uppercase) | |
timestamp | ticker timestamp provided by exchange in microseconds since epoch | |
local_timestamp | ticker message arrival timestamp in microseconds since epoch | |
type | option type, possible values:
| |
strike_price | option strike price | |
expiration | option expiration date in microseconds since epoch | |
open_interest | current open interest, empty is exchange does not provide one | |
last_price | price of the last trade, empty if there weren't any trades yet | |
bid_price | current best bid price, empty if there aren't any bids | |
bid_amount | current best bid amount, empty if there aren't any bids | |
bid_iv | implied volatility for best bid, empty if there aren't any bids | |
ask_price | current best ask price, empty if there aren't any asks | |
ask_amount | current best ask amount, empty if there aren't any asks | |
ask_iv | implied volatility for best ask, empty if there aren't any asks | |
mark_price | mark price, empty is exchange does not provide one | |
mark_iv | implied volatility for mark price, empty is exchange does not provide one | |
underlying_index | underlying index name that option contract is based upon | |
underlying_price | underlying price, empty is exchange does not provide one | |
delta | delta value for the option, empty is exchange does not provide one | |
gamma | gamma value for the option, empty is exchange does not provide one | |
vega | vega value for the option, empty is exchange does not provide one | |
theta | theta value for the option, empty is exchange does not provide one | |
rho | rho value for the option, empty is exchange does not provide one | |
exchange | symbol | timestamp | local_timestamp | type | strike_price | expiration | open_interest | last_price | bid_price | bid_amount | bid_iv | ask_price | ask_amount | ask_iv | mark_price | mark_iv | underlying_index | underlying_price | delta | gamma | vega | theta | rho |
deribit | BTC-9JUN20-9875-P | 1591574399413000 | 1591574400196008 | put | 9875 | 1591689600000000 | 0.1 | 0.0295 | 0.0205 | 15.0 | 55.91 | 0.0235 | 15.0 | 68.94 | 0.02210436 | 62.89 | SYN.BTC-9JUN20 | 9756.36 | -0.61752 | 0.00103 | 2.24964 | -53.05655 | -0.22796 |
deribit | BTC-9JUN20-9875-P | 1591574404454000 | 1591574404473112 | put | 9875 | 1591689600000000 | 0.1 | 0.0295 | 0.0205 | 15.0 | 55.91 | 0.0235 | 15.0 | 68.94 | 0.02209480 | 62.86 | SYN.BTC-9JUN20 | 9756.37 | -0.61757 | 0.00103 | 2.24954 | -53.02754 | -0.22798 |
deribit | BTC-9JUN20-9875-C | 1591574397505000 | 1591574400196010 | call | 9875 | 1591689600000000 | 44.3 | 0.0080 | 0.0095 | 0.5 | 61.00 | 0.0105 | 20.0 | 65.33 | 0.00992836 | 62.87 | SYN.BTC-9JUN20 | 9756.25 | 0.38232 | 0.00103 | 2.24933 | -53.03038 | 0.13272 |
deribit | BTC-9JUN20-9750-C | 1591574399414000 | 1591574400196011 | call | 9750 | 1591689600000000 | 30.5 | 0.0145 | 0.0145 | 0.3 | 58.80 | 0.0160 | 20.0 | 65.02 | 0.01527998 | 62.05 | SYN.BTC-9JUN20 | 9756.36 | 0.51442 | 0.00109 | 2.35092 | -54.69903 | 0.17789 |
deribit | BTC-9JUN20-9750-P | 1591574397562000 | 1591574400196012 | put | 9750 | 1591689600000000 | 0.8 | 0.0185 | 0.0140 | 0.3 | 59.40 | 0.0155 | 0.3 | 65.63 | 0.01464260 | 62.06 | SYN.BTC-9JUN20 | 9756.25 | -0.48570 | 0.00109 | 2.35092 | -54.70775 | -0.17832 |
deribit | BTC-9JUN20-9625-P | 1591574397824000 | 1591574400197202 | put | 9625 | 1591689600000000 | 9.5 | 0.0130 | 0.0090 | 0.4 | 61.64 | 0.0105 | 0.4 | 68.29 | 0.00975848 | 65.01 | SYN.BTC-9JUN20 | 9756.25 | -0.35780 | 0.00097 | 2.20136 | -53.66975 | -0.13100 |
deribit | BTC-9JUN20-9625-C | 1591574397359000 | 1591574400197208 | call | 9625 | 1591689600000000 | 18.0 | 0.0220 | 0.0215 | 15.0 | 57.39 | 0.0235 | 20.0 | 66.31 | 0.02320750 | 65.02 | SYN.BTC-9JUN20 | 9756.18 | 0.64212 | 0.00097 | 2.20150 | -53.67532 | 0.22058 |
deribit | BTC-9JUN20-9500-P | 1591574397940000 | 1591574400197209 | put | 9500 | 1591689600000000 | 51.5 | 0.0065 | 0.0060 | 0.5 | 66.31 | 0.0065 | 17.1 | 68.91 | 0.00625625 | 67.64 | SYN.BTC-9JUN20 | 9756.30 | -0.25091 | 0.00080 | 1.87744 | -47.62196 | -0.09165 |
deribit | BTC-9JUN20-9500-C | 1591574399413000 | 1591574400197211 | call | 9500 | 1591689600000000 | 43.8 | 0.0165 | 0.0315 | 20.0 | 62.19 | 0.0350 | 0.4 | 80.06 | 0.03252520 | 67.64 | SYN.BTC-9JUN20 | 9756.36 | 0.74914 | 0.00080 | 1.87725 | -47.61618 | 0.25540 |
https://datasets.tardis.dev/v1/deribit/options_chain/2020/03/01/OPTIONS.csv.gz
Deribit options chain for 2020-03-01
https://datasets.tardis.dev/v1/okex-options/options_chain/2020/03/01/OPTIONS.csv.gz
OKEx options chain for 2020-03-01
Top of the book (best bid/ask) data reconstructed from exchanges' real-time WebSocket order book L2 data feeds. - best bid/ask recorded every time top of the book has changed.
We on purpose choose this solution over native exchanges real-time quotes feeds as those vary a lot between exchanges, can be throttled, some are absent at all, often are delayed and published in batches in comparison to more granular L2 updates which are the basis for our quotes dataset.
CSV quotes schema
dataset preview
column name | description | |
exchange | | |
symbol | instrument symbol as provided by exchange (always uppercase) | |
timestamp | timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback | |
local_timestamp | message arrival timestamp in microseconds since epoch | |
ask_amount | best ask amount as provided by exchange, empty if there aren't any asks | |
ask_price | best ask price as provided by exchange, empty if there aren't any asks | |
bid_price | best bid price as provided by exchange, empty if there aren't any bids | |
bid_amount | best bid amount as provided by exchange, empty if there aren't any bids | |
exchange | symbol | timestamp | local_timestamp | ask_amount | ask_price | bid_price | bid_amount |
huobi-dm-swap | BTC-USD | 1585699201147000 | 1585699201270777 | 86 | 6423 | 6422.9 | 112 |
huobi-dm-swap | BTC-USD | 1585699201175000 | 1585699201292111 | 86 | 6423 | 6422.9 | 114 |
huobi-dm-swap | BTC-USD | 1585699201257000 | 1585699201373479 | 84 | 6423 | 6422.9 | 219 |
huobi-dm-swap | BTC-USD | 1585699201279000 | 1585699201495667 | 64 | 6423 | 6422.9 | 219 |
huobi-dm-swap | BTC-USD | 1585699201295000 | 1585699201495715 | 64 | 6423 | 6422.9 | 229 |
huobi-dm-swap | BTC-USD | 1585699201447000 | 1585699201564788 | 2 | 6423 | 6422.9 | 229 |
huobi-dm-swap | BTC-USD | 1585699201556000 | 1585699201677770 | 64 | 6423 | 6422.9 | 229 |
huobi-dm-swap | BTC-USD | 1585699201668000 | 1585699201784213 | 64 | 6423 | 6422.9 | 235 |
huobi-dm-swap | BTC-USD | 1585699201747000 | 1585699201865051 | 2 | 6423 | 6422.9 | 235 |
https://datasets.tardis.dev/v1/huobi-dm-swap/quotes/2020/05/01/BTC-USD.csv.gz
Huobi DM Swap BTC-USD quotes for 2020-05-01
https://datasets.tardis.dev/v1/options/quotes/2020/05/01/OPTIONS.csv.gz
datasets.tardis.dev
Deribit OPTIONS instruments quotes for 2020-05-01
book_ticker
Derivative instrument ticker info (open interest, funding, mark price, index price) collected from exchanges' real-time WebSocket instruments & tickers data feeds.
Anytime any of the tracked values has changed data was added to final dataset.
CSV derivative_ticker schema
dataset preview
column name | description | |
exchange | | |
symbol | instrument symbol as provided by exchange (always uppercase) | |
timestamp | timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback | |
local_timestamp | message arrival timestamp in microseconds since epoch | |
funding_timestamp | timestamp of the next funding event in microseconds since epoch, empty if exchange does not provide one | |
funding_rate | funding rate that will take effect on the next funding event at funding timestamp, for some exchanges it's fixed, for other it fluctuates, empty if exchange does not provide one | |
predicted_funding_rate | estimated predicted funding rate for the next after closest funding event, empty if exchange does not provide one | |
open_interest | current open interest, empty if exchange does not provide one | |
last_price | last instrument price, empty if exchange does not provide one | |
index_price | index price of the instrument, empty if exchange does not provide one | |
mark_price | mark price of the instrument, empty if exchange does not provide one | |
1 | exchange | symbol | timestamp | local_timestamp | funding_timestamp | funding_rate | predicted_funding_rate | open_interest | last_price | index_price | mark_price |
2 | bitmex | ETHUSD | 1585699199651000 | 1585699202577291 | 1585713600000000 | 0.0001 | 0.001654 | 45921455 | 133.25 | 133.14 | 133.15 |
3 | bitmex | ETHUSD | 1585699200000000 | 1585699204834359 | 1585713600000000 | 0.0001 | 0.001654 | 45921455 | 133.25 | 133.12 | 133.13 |
4 | bitmex | ETHUSD | 1585699202925000 | 1585699205076090 | 1585713600000000 | 0.0001 | 0.001654 | 45921455 | 133.3 | 133.12 | 133.13 |
5 | bitmex | ETHUSD | 1585699202925000 | 1585699205090339 | 1585713600000000 | 0.0001 | 0.001654 | 45883853 | 133.3 | 133.12 | 133.13 |
6 | bitmex | ETHUSD | 1585699203465000 | 1585699205274555 | 1585713600000000 | 0.0001 | 0.001654 | 45883853 | 133.25 | 133.12 | 133.13 |
7 | bitmex | ETHUSD | 1585699204439000 | 1585699205951209 | 1585713600000000 | 0.0001 | 0.001654 | 45883853 | 133.15 | 133.12 | 133.13 |
8 | bitmex | ETHUSD | 1585699205000000 | 1585699206389317 | 1585713600000000 | 0.0001 | 0.001654 | 45883853 | 133.15 | 133.09 | 133.1 |
9 | bitmex | ETHUSD | 1585699207279000 | 1585699207490211 | 1585713600000000 | 0.0001 | 0.001654 | 45883853 | 133.2 | 133.09 | 133.1 |
10 | bitmex | ETHUSD | 1585699207279000 | 1585699208084951 | 1585713600000000 | 0.0001 | 0.001654 | 45867677 | 133.2 | 133.09 | 133.1 |
https://datasets.tardis.dev/v1/bitmex/derivative_ticker/2019/04/01/ETHUSD.csv.gz
BitMEX ETHUSD derivative ticker for 2019-04-01
https://datasets.tardis.dev/v1/ftx/derivative_ticker/2019/04/01/PERPETUALS.csv.gz
datasets.tardis.dev
FTX PERPETUALS instruments derivative ticker for 2019-04-01
CSV liquidations schema
dataset preview
column name | description | |
exchange | | |
symbol | instrument symbol as provided by exchange (always uppercase) | |
timestamp | timestamp provided by exchange in microseconds since epoch - if exchange does not provide one local_timestamp value is used as a fallback | |
local_timestamp | message arrival timestamp in microseconds since epoch | |
id | liquidation id as provided by exchange, empty if exchange does not provide one - different exchanges provide id's as numeric values, GUID's or other strings, and some do not provide that information at all | |
side | liquidation side:
| |
price | liquidation price as provided by exchange | |
amount | liquidation amount as provided by exchange | |
exchange | symbol | timestamp | local_timestamp | id | side | price | amount |
binance-futures | BTCUSDT | 1632009737493000 | 1632009737505152 | | sell | 48283.81 | 0.01 |
binance-futures | BTCUSDT | 1632009802385000 | 1632009802398690 | | buy | 48339.11 | 0.132 |
binance-futures | BTCUSDT | 1632009870475000 | 1632009870485139 | | buy | 48337.02 | 0.004 |
binance-futures | BTCUSDT | 1632009889760000 | 1632009889784144 | | buy | 48346.54 | 0.002 |
binance-futures | BTCUSDT | 1632009891282000 | 1632009891296156 | | buy | 48350.34 | 0.032 |
binance-futures | BTCUSDT | 1632009892636000 | 1632009892646433 | | buy | 48355.68 | 0.001 |
binance-futures | BTCUSDT | 1632009970533000 | 1632009970544039 | | buy | 48290.57 | 0.042 |
binance-futures | BTCUSDT | 1632010836285000 | 1632010836297995 | | sell | 48186.15 | 0.036 |
binance-futures | BTCUSDT | 1632010899415000 | 1632010899428203 | | sell | 48150.64 | 0.265 |
https://datasets.tardis.dev/v1/ftx/liquidations/2021/09/01/PERPETUALS.csv.gz
FTX perpetual futures liquidations for 2021-09-01
https://datasets.tardis.dev/v1/bitmex/liquidations/2021/09/01/XBTUSD.csv.gz
BitMEX XBTUSD liquidations for 2021-09-91
In addition to standard currency pairs & instrument symbols that can be requested when via CSV datasets API, each exchange has additional special grouped symbols available depending if it supports given market type: SPOT, FUTURES, OPTIONS and PERPETUALS. When such symbol is requested then downloaded file for it has all the data for all instruments belonging for given market type.
This is especially useful for options instruments that as specifying each option symbol one by one can be mundane process, using 'OPTIONS' as a symbol gives data for all options available at given time.
- Examples:
- all downloadable datasets are gzip compressed
- historical market data is available in daily intervals (separate file for each day) based on local timestamp (timestamp of message arrival) split by exchange, data type and symbol
- data for a given day is available on the next day around 3h after 00:00 UTC - exact date until when data is available can be requested via /exchanges/:exchange API call (
datasets.exportedUntil
), e.g., https://api.tardis.dev/v1/exchanges/ftx - datasets are ordered and split into separate daily files by
local_timestamp
(timestamp of message arrival time) - empty gzip compressed file is being returned in case of no data available for a given day, symbol and data type, e.g., exchange downtime, very low volume currency pairs etc.
- if
timestamp
equals tolocal_timestamp
it means that exchange didn't provide timestamp for message, e.g., BitMEX order book updates - cell in CSV file is empty if there's no value for it, e.g., no trade id if a given exchange doesn't provide one
- datasets are sourced from Tardis.dev HTTP API, which in turn provides the the data sourced from exchanges real-time WebSocket market data feeds (in contrast to REST API endpoints)
- See "Data FAQ" regarding potential order book overlaps issues, non monotonically increasing exchanges timestamps, duplicated trade data and more
Historical datasets for the first day of each month are available to download without API key.
Python
Node.js
# pip install tardis-dev
# requires Python >=3.6
from tardis_dev import datasets, get_exchange_details
import logging
# comment out to disable debug logs
logging.basicConfig(level=logging.DEBUG)
# function used by default if not provided via options
def default_file_name(exchange, data_type, date, symbol, format):
return f"{exchange}_{data_type}_{date.strftime('%Y-%m-%d')}_{symbol}.{format}.gz"
# customized get filename function - saves data in nested directory structure
def file_name_nested(exchange, data_type, date, symbol, format):
return f"{exchange}/{data_type}/{date.strftime('%Y-%m-%d')}_{symbol}.{format}.gz"
# returns data available at https://api.tardis.dev/v1/exchanges/deribit
deribit_details = get_exchange_details("deribit")
# print(deribit_details)
datasets.download(
# one of https://api.tardis.dev/v1/exchanges with supportsDatasets:true - use 'id' value
exchange="deribit",
# accepted data types - 'datasets.symbols[].dataTypes' field in https://api.tardis.dev/v1/exchanges/deribit,
# or get those values from 'deribit_details["datasets"]["symbols][]["dataTypes"] dict above
data_types=["incremental_book_L2", "trades", "quotes", "derivative_ticker", "book_snapshot_25", "book_snapshot_5", "liquidations"],
# change date ranges as needed to fetch full month or year for example
from_date="2019-11-01",
# to date is non inclusive
to_date="2019-11-02",
# accepted values: 'datasets.symbols[].id' field in https://api.tardis.dev/v1/exchanges/deribit
symbols=["BTC-PERPETUAL", "ETH-PERPETUAL",],
# (optional) your API key to get access to non sample data as well
api_key="YOUR API KEY",
# (optional) path where data will be downloaded into, default dir is './datasets'
# download_dir="./datasets",
# (optional) - one can customize downloaded file name/path (flat dir strucure, or nested etc) - by default function 'default_file_name' is used
# get_filename=default_file_name,
# (optional) file_name_nested will download data to nested directory structure (split by exchange and data type)
# get_filename=file_name_nested,
)
If you're running into
RuntimeError: This event loop is already running
error try solution from https://github.com/ipython/ipython/issues/11338#issuecomment-646539516 (adding nest_asyncio).// npm install [email protected]
// requires node version >=12
// remove it to disable debug logs
process.env.DEBUG = 'tardis-dev*'
const { downloadDatasets, getExchangeDetails } = require('tardis-dev')
;(async () => {
// returns data available at https://api.tardis.dev/v1/exchanges/deribit
const deribitDetails = await getExchangeDetails('deribit')
// console.log(deribitDetails.datasets)
await downloadDatasets({
exchange: 'deribit', // one of https://api.tardis.dev/v1/exchanges with supportsDatasets:true - use 'id' value
dataTypes: ['incremental_book_L2', 'trades', 'quotes', 'derivative_ticker', 'book_snapshot_25', 'book_snapshot_5', 'liquidations'], // accepted data types - 'datasets.symbols[].dataTypes' field in https://api.tardis.dev/v1/exchanges/deribit, or get those values from 'deribitDetails.datasets.symbols[].dataTypes' object above
from: '2019-11-01', // change date ranges as needed to fetch full month or year for example
to: '2019-11-02', // to date is non inclusive
symbols: ['BTC-PERPETUAL', 'ETH-PERPETUAL'], // accepted values: 'datasets.symbols[].id' field in https://api.tardis.dev/v1/exchanges/deribit, or `deribitDetails.datasets.symbols[].id` from object above
apiKey: 'YOUR_API_KEY', // (optional) your API key to get access to non sample data as well
// downloadDir:'./datasets', // (optional) path where data will be downloaded into, default dir is './datasets'
// getFilename: getFilenameDefault, // (optional) - one can customize downloaded file name/path (flat dir strucure, or nested etc) - by default function 'getFilenameDefault' is used
// getFilename: getFilenameCustom // (optional) getFilenameCustom will download data to nested directory structure (split by exchange and data type)
})
})().catch((e) => {
console.log('download error', e)
})
// function used by default if not provided via options
function getFilenameDefault({ exchange, dataType, format, date, symbol }) {
return `${exchange}_${dataType}_${date.toISOString().split('T')[0]}_${symbol}.${format}.gz`
}
// customized get filename function - saves data in nested directory structure
function getFilenameCustom({ exchange, dataType, format, date, symbol }) {
return `${exchange}/${dataType}/${date.toISOString().split('T')[0]}_${symbol}.${format}.gz`
}
get
https://datasets.tardis.dev/v1
/:exchange/:dataType/:year/:month/:day/:symbol.csv.gz
Datasets API reference
- symbols param provided to datasets API in comparison to HTTP API needs to be both always uppercase and have '/' and ':' characters replaced with '-' so symbol is url safe.
- list of allowed symbols for each exchange can be requested via /exchanges/:exchange API call, e.g., https://api.tardis.dev/v1/exchanges/deribit -
datasets.symbols[].id
field
Last modified 11mo ago