Original link: https://xuanwo.io/reports/2022-38/
Iteration 22 starts on 9/12 and ends on 9/25 for two weeks. My main work in this cycle is to delete code:
After several iterations, Databend has completed the refactoring of New Executor, New Planner, and is currently working on New Expression. With the addition of new features such as official support for JOIN, Databend’s code base has also experienced serious corruption: outdated codes are scattered in every corner of the project, and many new function implementations still rely on deprecated ones. logic. I started by cleaning up the Old Planner to try to alleviate this problem: the initial plan was just to fight for three days to remove the dependency on sqlparser-rs , but in the end I found that Databend’s dependency on it was so deep that it became spanning several Large jobs with iteration cycles.
A long time ago, I added See you again : refactor: Dedicate See you again to the old planner in DfStatement . By introducing a Dummy Statement SeeYouAgain in the old DfStatement, most of the operations were directly forwarded to the New Planner. Now I started a new project: refactor: Old Planner Never See Again (Part 1) . This project aims to completely remove Old Planner and completely clean up outdated code. When I implemented Part 2, I found that things were not simple. Many complex issues were intertwined. In the past, complex dependencies that were not exposed in the same crate were exposed at once. In order to delete the old code smoothly, I was forced to do a lot of foreshadowing:
- refactor: Rename planners to legacy-planners : Rename Old Planner
- refactor: Make planner depends on TableContext trait : remove Planner’s dependency on QueryContext
- refactor: Move optimizer into planner : Move the Optimizer into the Planner, removing the dependency on
query/sql - refactor: Cleanup planner to make space for saying goodbye : Cleanup the code in the Planner and remove the dependency on the Cluster specific structure
- refactor: Use String to replace Expression in DeletePlan : Avoid relying on Expression directly in Plan
- refactor(planner): Move all metadata out : Move the structure in the query to the external crate
- refactor: Move plans from old planner to new : Move plans from Old Planner to New Planner
- refactor: Remove the old planner’s interpreter creation : completely remove logic that is no longer used
- refactor: Old Planner Never See Again (Part 2) : Completely removed all PlanNode and PlanParser
- refactor: Delay physical scalar build to interpreter : Do not perform PhysicalScalar build in Bind stage
- refactor: Rename Expression to LegacyExpression and move out : Remove the Old Expression
After doing all of the above, I’m only halfway done removing sqlparser-rs , and next I need:
- Adjust the Table API so that it no longer depends on any Plan, so that the catalog crate does not need to depend on the planner (because the planner depends on the catalog)
- When adjusting Fuse Table, build Expression based on Scalar instead of Binder
- Adjust TableScan PhysicalPlan so that it no longer depends on ReadDataSourcePlan
- Adjust ReadDataSourcePlan so that it no longer depends on Expression
- …
I hope to complete these tasks in the next cycle, and officially say goodbye to Old Planner~
This article is reproduced from: https://xuanwo.io/reports/2022-38/
This site is for inclusion only, and the copyright belongs to the original author.