Random Facts

I put random facts I find interesting in here, or something I want to remember.

Latency and A/B Testing

A famous Amazon A/B test intentionally delayed page latency by 100ms and found even small delays (not noticeable by blink of an eye) can have substantial impact on user engagement and revenue outcomes. This is also an example of high elasticity, where demand is very sensitive to small factors. If a site slows just slightly, there's a big shift in the user response and behavior. Thus, there is a high elasticity of demand with respect to latency.

Elasticity

Elasticity can be thought of as synonymous with sensitivity. Other economic examples of elasticity include price elasticity where raising the price of a product like coffee by 10 cents can have dramatic reduction in the number of sales.

The mathematical formulation of elasticity is:

$$\text{Price elasticity} = \frac{\%\Delta \text{Quantity Demanded}}{\%\Delta \text{Price}}$$

For elasticity $>1$ it is elastic (sensitive), elasticity $<1$ it is inelastic (insensitive) and $=1$ it is unit elastic (proportional).

Power Law

The power law comes up quite a bit when it comes to discussing the scalability of AI performance (as well as its meaning in data vis). That's why it's important to understand the mathematics behind it, how to spot a power law, and what it really implies.

A power law defines the relationship between two quantities where one is proportional to a power of another, $Y = kX^{\alpha}$. A small change in $X$ can lead to a proportionally large change in $Y$. You can have positive alpha for explosive growth or negative alpha for inverse shrinking.

One attribute of power law is scale invariance where $f(\lambda x) = \lambda^{\Delta} f(x)$. Scaling $x$ by $c$ results in $c^{\alpha}$ in the $Y$. Consequently, the distribution or relationship looks the same regardless of the scale you use.

Power Law Distribution

A power law distribution is one where the probability of an event is proportional to the power of the magnitude of the event, so $p(\text{event}) = k \cdot \text{magnitude}^{-\alpha}$ where $\alpha > 0$. Note that the convention is for a negative alpha because the probability decays as a power of $x$. Compared to a normal distribution, a power law distribution has a fat tail so the probability of extreme events is higher than a normal distribution. An example is in the startup world where the unicorns dominate wealth creation and return on investment. The average case does not matter, because all the value comes from the tail.

AI Power Law

In the context of AI you have the power law relationship between loss and compute such that $L(C) \approx kC^{-\alpha}$. On a log to log plot, it will be a straight line or linear relationship. But in actually loss quantities, the reduction is sublinear, isn't that bad? No - the inverse power law here means that you can always keep reducing the loss although the gains get less with the flattening of the curve. With more compute you will always get predictable linear gain in the log to log scale!

Free Cash Flow

Recently the news of Oracle's stock rally read:

The biggest question for investors is how Oracle can continue funding that accelerating capex spend. Already, Oracle’s free cash flow in fiscal 2025 was negative for the first time in over a decade, as its executives doubled down on their cloud dreams. And despite the recent stock surge, Oracle might be reticent to issue more equity, considering it has been buying back its shares for years.

Free cash flow (FCF) is the amount of cash left over after paying for operating expenditure (opex) and capital expenditure (capex). If the free cash flow is positive, the company has spare cash left over to pay dividends, buy back stock, pay debt or make new investments. If the free cash flow is negative, the company does not have enough cash to cover all its expenses and need to find money via borrowing, issuing stock etc. to make those payments.

Issuing More Equity = Bad?

Issuing stock is a way for companies to raise money, but at the expense of shareholders who see the value of their shares diluted.

The opposite of this is buying back stock which reduces outstanding shares. This increase Earnings Per Share (EPS) as profit is distributed over fewer shares. This also signals confidence to investors that they think the stock is undervalued and rewards investors.

Vesting

Think of vesting as a gift with strings attached (conditions to meet). For example with RSUs you are promised a certain amount of stock, but you only get them if you stay for long enough periods of time at the company. The first year you might get 250, then 500 the next, then 1000 the year after. If they are vested, it means that you've met the conditions and gained ownership over it. If they are not vested, you stand to lose them.

When RSUs are vested, they turn into actual shares of the company that you now own. This is treated by the IRS as income. This can be tricky because taxes are due even when you do not sell! Commonly companies will sell shares automatically to cover this i.e. withhold the taxes for you when vesting.

Stock Options

Stock options differ from RSUs in that you get the opportunity to buy stock at a set price, aka the strike price. The strike price allows you to lock in the stock price at the current rate so that if the company does well in the future and the stock rises, you can buy the stock at the lower guaranteed rate and sell to pocket the difference. Note that it is not a discount upfront, you only earn money if the stock grows.