Robust Python Pipeline for Binance Data

@sajjadtaghvaeifr

Hello, I trust you're doing well. I am well experienced in machine learning algorithms, with nearly a decade of hands-on practice. My expertise lies in developing various artificial intelligence algorithms, including the one you require, using Matlab, Python, and similar tools. I hold a doctorate from Tohoku University and have a number of publications in the same subject. My portfolio, which showcases my past work, is available for your review. Your project piqued my interest, and I would be delighted to be part of it. Let's connect to discuss in detail. Warm regards. please check my portfolio link: https://www.freelancer.com/u/sajjadtaghvaeifr

@Web1devloper

As an experienced data analyst and scientist with a deep understanding of Python, I am perfectly equipped to deliver you a robust and efficient pipeline for processing Binance data à la your requirements. My 8+ years in the field have familiarized me with working on complex datasets and implementing advanced analytics techniques essential for this project. My expertise extends from core data handling - including setting up local storage solutions, aligning different frequencies of data, to microstructure feature extraction, and advanced optimization for ML readiness. I excel at preventing lookahead biases, normalizing features, labeling, and ensuring outputs that are clean, reliable-as-an-ox Parquet files promptly available for ML model training. Having successfully rolled out solutions across finance amongst other industries - à la businesses dependent on data management and process optimization - I've honed my abilities to yield accurate results amidst tight deadlines. With me at the helm of your project, you can rest assured about quality, efficiency, and unlocking the full potential of your Binance data. Let's embark on this data-driven journey together.

@toqeer74

Addressing the core challenge of efficiently processing and normalizing large datasets without introducing lookahead bias begins with a robust ingestion and database setup. Programmatic downloads of Binance's historical data can be executed seamlessly with the appropriate use of Python libraries, leveraging DuckDB for its high-performance data handling capabilities. Prioritizing the alignment of varying data frequencies into a unified timestamp sequence is crucial for accurate feature extraction. I will implement advanced statistical techniques using Polars for processing and ensure the extracted features are immediately ML-ready. The initial deliverable will be ready in 14 days. Ready to kick this off, what's the best way to get started?

@collenliebenberg

I see you need a robust Python pipeline to process Binance Futures historical data into an ML-ready format. I'd build this using Python scripts to ingest and process data from Binance Vision, ensuring clean, normalized features stored in Parquet format. This will allow you to efficiently analyze and model data for better decision-making. I've worked with similar data pipelines for finance and trading industries, ensuring accurate results. Quick question: How soon do you need this pipeline up and running? Regards, Collen Jr Liebenberg