<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->

# Apache DataFusion 54.1.0 Changelog

This release consists of 19 commits from 9 contributors. See credits at the end of this changelog for more information.

See the [upgrade guide](https://datafusion.apache.org/library-user-guide/upgrading.html) for information on how to upgrade from previous versions.

**Documentation updates:**

- [branch-54] Add datafusion.execution.enable_file_stream_work_stealing config [#23296](https://github.com/apache/datafusion/pull/23296) (andygrove)

**Other:**

- [branch-54] fix: preserve null_aware on logical JoinNode proto round-trip (backport #22104) [#22785](https://github.com/apache/datafusion/pull/22785) (mithuncy)
- [branch-54]: backport #22811 (bugfix: changed return type of spark's width_bucket to i64) [#23087](https://github.com/apache/datafusion/pull/23087) (mbutrovich)
- [branch-54] backport #22857 (Skip loading Parquet page index when row-group statistics already prove it cannot prune) [#23088](https://github.com/apache/datafusion/pull/23088) (mbutrovich)
- [branch-54] backport #23192 `array_compact` handle edge case with NULLs [#23196](https://github.com/apache/datafusion/pull/23196) (comphead)
- [branch-54] fix: Avoid panicing when stats are not available for a file group split (backport #23277) [#23340](https://github.com/apache/datafusion/pull/23340) (mkleen)
- [branch-54] fix: `approx_distinct` over-counts for utf8view (backport #22815, adapted) [#23576](https://github.com/apache/datafusion/pull/23576) (mbutrovich)
- [branch-54] fix: isolate anonymous file statistics cache (backport #22950, adapted) [#23573](https://github.com/apache/datafusion/pull/23573) (mbutrovich)
- [branch-54] fix: `= ANY (SELECT ...)` / `<> ALL (SELECT ...)` schema error (backport #22915) [#23575](https://github.com/apache/datafusion/pull/23575) (mbutrovich)
- [branch-54] fix: NestedLoopJoinExec emits spurious unmatched-left rows with multiple probe partitions (backport #22791) [#23577](https://github.com/apache/datafusion/pull/23577) (mbutrovich)
- [branch-54] fix: regex simplification of anchored patterns produces wrong results (backport #22727) [#23578](https://github.com/apache/datafusion/pull/23578) (mbutrovich)
- [branch-54] fix: Correctly compute nullability in recursive CTE schemas (backport #22552) [#23579](https://github.com/apache/datafusion/pull/23579) (mbutrovich)
- [branch-54] fix: handle `IS TRUE` correctly in `EliminateOuterJoin` (backport #22444) [#23580](https://github.com/apache/datafusion/pull/23580) (mbutrovich)
- [branch-54] fix: preserve no-filter SMJ matches across pending outer batches (backport #23049) [#23574](https://github.com/apache/datafusion/pull/23574) (mbutrovich)
- [branch-54] perf: avoid intermediate slice allocation in Spark slice function (backport #23481) [#23582](https://github.com/apache/datafusion/pull/23582) (mbutrovich)
- [branch-54] fix: don't duplicate volatile expressions when pushing projection into file scan (backport #23395, adapted) [#23585](https://github.com/apache/datafusion/pull/23585) (fordN)
- [branch-54] chore: fix cargo audit [#23607](https://github.com/apache/datafusion/pull/23607) (alamb)
- [branch-54] Resolve lost wakeup in SpillPoolReader with multiple conc… [#23654](https://github.com/apache/datafusion/pull/23654) (pepijnve)
- [branch-54] Handle nulls in type coercion of higher-order UDFs, map_extract, spark array_repeat (backport #23071) [#23629](https://github.com/apache/datafusion/pull/23629) (gstvg)

## Credits

Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor.

```
    11	Matt Butrovich
     1	Andrew Lamb
     1	Andy Grove
     1	Ford
     1	Michael Kleen
     1	Mithun Chicklore Yogendra
     1	Oleks V
     1	Pepijn Van Eeckhoudt
     1	gstvg
```

Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release.
