Skip to content

Futuristic pipeline

Jordan Petridis requested to merge futuristic-pipeline into master

Couple things have happened here.

First of all the pipeline switched from a synchronous + multithreaded(with rayon) approach to a (wanna be)"async" with cpupool offloading. The futures implementation is kinda a mess, but it seems to be functional and on par with the performance of the synchronous code. I am gonna continue improving it as I learn more of the framework.

Secondly, there was an enchantment of the test suit. Added a bunch of extra test for the insertable Diesel models and swithed to using internet archive snapshots from RSS Feeds.

Third, the we switched to the Criterion crate for benchmarking. It can be built on stable too.

Last but not least, there was a module split of the Diesel models. Now each "Category" of a model like "Insertables of the Episode table" have their own NewEpisode.rs module instead of 2 insertable/queryables modules.

Merge request reports