Moment of introspection aside, I’m not sure what the future holds for agents and generative AI. My use of agents has proven to have significant utility (for myself at the least) and I have more-than-enough high-impact projects in the pipeline to occupy me for a few months. Although certainly I will use LLMs more for coding apps which benefit from this optimization, that doesn’t imply I will use LLMs more elsewhere: I still don’t use LLMs for writing — in fact I have intentionally made my writing voice more sardonic to specifically fend off AI accusations.
Many people reading this will call bullshit on the performance improvement metrics, and honestly, fair. I too thought the agents would stumble in hilarious ways trying, but they did not. To demonstrate that I am not bullshitting, I also decided to release a more simple Rust-with-Python-bindings project today: nndex, an in-memory vector “store” that is designed to retrieve the exact nearest neighbors as fast as possible (and has fast approximate NN too), and is now available open-sourced on GitHub. This leverages the dot product which is one of the simplest matrix ops and is therefore heavily optimized by existing libraries such as Python’s numpy…and yet after a few optimization passes, it tied numpy even though numpy leverages BLAS libraries for maximum mathematical performance. Naturally, I instructed Opus to also add support for BLAS with more optimization passes and it now is 1-5x numpy’s speed in the single-query case and much faster with batch prediction. 3 It’s so fast that even though I also added GPU support for testing, it’s mostly ineffective below 100k rows due to the GPU dispatch overhead being greater than the actual retrieval speed.
，这一点在爱思助手下载最新版本中也有详细论述
Что думаешь? Оцени!
最尴尬的还是我们广大消费者。当你作为新手还在驾校为自动挡还是手动挡如何选择而头疼的时候，当你还没有体验过拉线油门、手刹漂移、小狗撒尿的驾驶乐趣的时候，厂商们却千方百计不想让你自己开车。