/2025/03/30/gettyimages-585201924-67e9638374896003219187.jpg?#)
/2025/03/30/un-jour-en-france-les-arbitres-face-aux-violences-dans-le-football-amateur-une-commune-privee-d-internet-un-tresor-en-dordogne-67e99ef882821616333506.jpg?#)
/2025/03/30/netflix-adolescence-la-serie-a-succes-qui-secoue-ses-spectateurs-67e9a321153a1866043896.jpg?#)
Show HN: Xorq – open-source Python-first Pandas-style pipelines
Hi HN, Dan, Hussain and Daniel here… After years of struggling with data pipelines that worked in notebooks but failed in production, we decided to do something about it. We created xorq to eliminate the constant headaches of SQL/pandas impedance mismatch, runtime debugging, wasteful recomputations and unreliable research-to-production deployments that plague traditional pandas-style pipeline workflows. xorq is built on Ibis and DataFusion.We’d love your feedback and contributions. xorq is [Apache 2.0 licensed](https://github.com/letsql/xorq/blob/main/LICENSE) to encourage open collaboration.Repo: https://github.com/letsql/xorqDocs: https://docs.xorq.devRoadmap Issues: https://github.com/letsql/xorqYou can get started `pip install xorq`.Or, if you use nix, you can simply run `nix run github:xorq-labs/xorq` and drop into an IPython shell.Demo video: https://youtu.be/jUk8vrR6bCwHere are some vignettes to look into next:1. MCP Server + Flight + XGBoost: https://docs.xorq.dev/vignettes/mcp_flight_server2. 1 DuckDB + 2 Writers + 1 Reader: https://docs.xorq.dev/vignettes/duckdb_concurrent3. OpenAI UDF: https://docs.xorq.dev/tutorials/hn_data_prepSome features to note:- Ibis-based multi-engine expression system: effortless engine-to-engine streaming- Cache expressions with `.cache` operator- Portable DataFusion-backed UDF engine with first class support for pandas dataframes- Serialize Expressions to and from YAML- Easily build Flight end-points by composing UDFsthanks for checking this out, and we’re here to answer any questions! Comments URL: https://news.ycombinator.com/item?id=43495811 Points: 8 # Comments: 0
Hi HN, Dan, Hussain and Daniel here… After years of struggling with data pipelines that worked in notebooks but failed in production, we decided to do something about it. We created xorq to eliminate the constant headaches of SQL/pandas impedance mismatch, runtime debugging, wasteful recomputations and unreliable research-to-production deployments that plague traditional pandas-style pipeline workflows. xorq is built on Ibis and DataFusion.
We’d love your feedback and contributions. xorq is [Apache 2.0 licensed](https://github.com/letsql/xorq/blob/main/LICENSE) to encourage open collaboration.
Repo: https://github.com/letsql/xorq
Docs: https://docs.xorq.dev
Roadmap Issues: https://github.com/letsql/xorq
You can get started `pip install xorq`.
Or, if you use nix, you can simply run `nix run github:xorq-labs/xorq` and drop into an IPython shell.
Demo video: https://youtu.be/jUk8vrR6bCw
Here are some vignettes to look into next:
1. MCP Server + Flight + XGBoost: https://docs.xorq.dev/vignettes/mcp_flight_server
2. 1 DuckDB + 2 Writers + 1 Reader: https://docs.xorq.dev/vignettes/duckdb_concurrent
3. OpenAI UDF: https://docs.xorq.dev/tutorials/hn_data_prep
Some features to note:
- Ibis-based multi-engine expression system: effortless engine-to-engine streaming
- Cache expressions with `.cache` operator
- Portable DataFusion-backed UDF engine with first class support for pandas dataframes
- Serialize Expressions to and from YAML
- Easily build Flight end-points by composing UDFs
thanks for checking this out, and we’re here to answer any questions!
Comments URL: https://news.ycombinator.com/item?id=43495811
Points: 8
# Comments: 0
