Skip to content

9.1.0
Compare
Choose a tag to compare
@ibis-project-bot ibis-project-bot released this 13 Jun 17:39
· 848 commits to main since this release
9.1.0
2dac5e4

9.1.0 (2024-06-13)

Features

  • all: enable passing in-memory data to create_table (#9251) (fa15c7d), closes #6593 #8863
  • api: add Table.value_counts for easy group by count on multiple fields (aba913d)
  • api: isoyear method (#9034) (4707c44)
  • api: support type arg to ibis.null() (8db686e)
  • api: support wider range of types in where arg to column reductions (582165f)
  • api: support wider range of types in where arg to table reductions (7aba385)
  • bigquery: implement a few URL ops (#9210) (3d0f9bc)
  • bigquery: support filtering by _TABLE_SUFFIX when using a wildcard table name (#9375) (62a25c4), closes #9371
  • datafusion: use pyarrow for type conversion (#9299) (5bef96a)
  • drop Python 3.9 and test on Python 3.10/3.12 (#9213) (c06285e)
  • duckdb: add catalog support to create_table (#9147) (07331b5)
  • duckdb: allow to use named in-memory db (#9241) (67460aa), closes #9240
  • duckdb: support and test 1.0 (#9297) (395c8b5)
  • pandas,dask: implement ops.StructColumn (#9302) (ea81d85)
  • polars: accept list of CSVs to read_csv (#9232) (7a272e3), closes #9230
  • polars: implement create_view/drop_view/drop_table (#9263) (c4324f5)
  • postgres: provide translation for hash ops (#9348) (57e2348)
  • pyarrow: support Arrow PyCapsule interface on ibis.Table objects (1a262b9)
  • pyspark: builtin udf support (#9191) (142c105)
  • pyspark: provide a mode option to manage both batch and streaming connections (e425ad5)
  • pyspark: support reading from and writing to Kafka (#9266) (1c7c6e3)
  • selectors: parse Python types in s.of_type (#9356) (c0ebdc8)
  • snowflake: implement array map and array filter (#9178) (9b42751)
  • snowflake: implement support for asof_join API (#9180) (49c6ce3)
  • snowflake: implement Table.sample (#9071) (307334b)
  • ux: improve error message on unequal schemas during set ops (#9115) (5488896)

Bug Fixes

  • api: treat col == None or col == ibis.NA as col.isnull() (#9114) (711bf9f)
  • bigquery: only register memtable if obj is not None (#9268) (f175d0a)
  • bigquery: quote all parts of table names (#9141) (e1338d5)
  • bigquery: quote qualified memtable names (#9149) (878d0d5)
  • bigquery: strip whitespace from bigquery field names (#9160) (8e5cc3b), closes #9112
  • clickhouse: more explicitly disallow null structs (#9305) (fc1d00f)
  • convert the uint64's from some backends' hash() to the desired int64 (900ecca)
  • datatypes: manually cast the type of pos to int16 for table.info() (#9139) (9eb1ed1)
  • datatypes: manually cast the type of pos to int16 for table.describe() (#9314) (c7fcddf)
  • ddl: use column names, not position, for insertion order (#9264) (3506f40)
  • deps: remove pydruid sqlalchemy dependency (#9092) (a0df103)
  • deps: update dependency datafusion to v37 (#9189) (49ecf8d)
  • deps: update dependency datafusion to v38 (#9278) (77aaecd)
  • deps: update dependency fsspec to <2024.5.1 (#9201) (15a5257)
  • deps: update dependency fsspec to <2024.6.1 (#9304) (d600a0d)
  • deps: update dependency sqlglot to >=23.4,<23.14 (#9118) (d8119fb)
  • deps: update dependency sqlglot to >=23.4,<23.15 (#9151) (ac2201d)
  • deps: update dependency sqlglot to >=23.4,<23.17 (#9209) (82a5f93)
  • deps: update dependency sqlglot to >=23.4,<23.18 (#9212) (b92dd7b)
  • deps: update dependency sqlglot to >=23.4,<24.2 (#9277) (98cb460)
  • deps: update dependency sqlglot to >=23.4,<25.2 (#9368) (d65a752)
  • deps: update dependency sqlglot to v24 (#9229) (a4918be)
  • deps: update dependency sqlglot to v25 (#9316) (2b921f8)
  • drop nulls in .collect() aggregation (b6e0c31)
  • duckdb: clean up temp view junk when using memtables in create_table (#9107) (4e7a00c)
  • duckdb: use existing table repr for settings view (#9155) (1892bfd)
  • exclude null values from first and last aggregations (22fffc7)
  • mysql: avoid creating any tables when using .sql() (#9363) (d2d5251), closes #9354
  • mysql: support parametrized datetime types (#9294) (ccfcbbc)
  • polars,mysql: avoid execution of query in _get_schema_using_query (#9290) (0348b9a)
  • pyspark: plumb through limit and params in export functions (1f36552)
  • replace NaNs with None in some backends when loading from pandas dataframe (#9094) (f2a7cd9), closes #9095 #8792
  • snowflake: ensure that timestamp conversion from parquet files is correct (#9181) (1ba4c32)
  • snowflake: properly pass schema and database for sqlglot generation (#9221) (1ecb319)
  • to_sql: use default backend for sql generation when set (#9228) (c66d6aa), closes #9227
  • trino: parse URL passed to ibis.connect (e3ee67b)
  • typing: map() can take ArrayValues not just ArrayColumns (#9282) (3ad1183)

Documentation

  • add API docs for operations (#9233) (11e0530)
  • add probabl podcast to 9.0 blog (#9105) (9d20b85)
  • api: document the ability to apply a different sort order using across (#9376) (f41c554)
  • blog: new blog post on sqlmesh + ibis (#9218) (8e015a7)
  • builtin: update url for packages.parquet file (#9132) (4f93a91)
  • descheming: move callout note outside of parameter description (#9133) (bb7bdb3), closes #8712
  • dev: update maintainers guide link (#9312) (379afac)
  • fix jupyterlite build (#9090) (63dcb92)
  • improve maintainers summit slides (#9207) (5999e2d)
  • presentation: minor updates to overview presentation (#9145) (f7c2dbb)
  • presentations: minor updates to history in overview presentation (#9341) (f5c7978)
  • presentation: updates to overview presentation (#9126) (2ba1884)
  • put DataType base class first in reference (eaf0e45)
  • pyspark: remove outdated connect() callout (#9327) (10112bd)
  • release blog for Ibis 9.0 (#8918) (0350815)
  • remove GitHub-specific Markdown in README.md (#9370) (ce0f1f2)
  • remove unrendered/unused top-level getting-started.qmd document (#9106) (66a67c0)
  • rework the homepage (#9088) (c68f9d4), closes #8856
  • search: append scraped API records to algolia index in CI (#9366) (05d9d7a)
  • talks: pycon 2024 maintainers talk (#9193) (77d6cb6)
  • update contribute index page content (#9349) (f130dae)
  • update the code of conduct link (#9337) (fa2de4d)
  • use interactive mode instead of execute for typed null docstrings (c27097b)
  • use more idiomatic group_by in readme example (#9307) (2aca613)
  • website: make icon grayscale for consistency (#9100) (fb81f92)

Refactors

  • deprecate fillna/dropna methods in favor of fill_null/drop_null (df0e656)
  • deprecate register api (#8863) (7a39bd3)
  • ir: actually remove analysis.py (#9087) (8508e3d)
  • pyspark: remove custom implementation of cursors (#9161) (9caa552)
  • remove ibisNA (#9344) (83db19d), closes #9311
  • snowflake: replace array repeat udf with builtin transform function (#9177) (b3abc9a)
  • sql: add LOWERED_OPS mapping for cleaner handling of operations implemented by "lowering" to simpler operations (7a9b4b6)
  • sql: extract aggregate handling out into common utility class (#9222) (56e0b38), closes #9170
  • sql: rename UNSUPPORTED_OPERATIONS to UNSUPPORTED_OPS for consistency (9e11957)
  • sql: use a rewrite rule to implement FillNa/DropNa (378251e)

Performance

  • api: rewrite union and intersection construction to support more operands (#9194) (5d7aa55)
  • api: speed up simple column accesses by avoiding dereferencing (#9156) (c770fa1)
Assets 3
Loading