📰 Summary (use your own words)
First ghost - its either significant or noise
First ghost - its either significant or noise
Explains GPT with a SQL demo
The Bayesian AB testing approach does mitigate the problem of peeking but it still increases the Type I error which is not the promise that the Bayesian approach promises. Conversely, because the frequentist approach is promising a Type I error rate in the form of p-value testing, it is explicitly breaking that promise if we peeked and took action on the experiments.
There is an infrastructure need to run and operate machine learning in production. The category of jobs related to this is labelled as MLOps or "Machine Learning Operations", but it is becoming apparent that there is a lot of overlap with the traditional data engineering roles.
Inspired by this https://python-data-science.readthedocs.io/en/latest/_images/architecture.png
- A way to evaluate model's design on limited data set
A way to subdivide a giant neural network into sub-networks which can be expertly trained on one aspect of the problem and then sparsely combined back together for each task.