Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Assume you need to execute the following query:

    SELECT SUM(A) FROM MyTable
For large tables it will take some time to compute the result. Now assume we append a new record and want to get the new result. The traditional approach is execute this query again. A better approach is to process this new record only by adding its value in A to the result of the previous query. It is important in (stateful) stream processing.

Something similar is implemented in these libraries which however rely on a different data processing conception (alternative to map-reduce):

https://github.com/asavinov/prosto - Functions matter! No join-groupby, No map-reduce.

https://github.com/asavinov/lambdo - Feature engineering and machine learning: together at last!



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: