The Hairy Chart, in Python

Project

Data Visualization

The federal funds rate and its forecast (hairy) paths, replicated in Python.

Author

Curtis Kephart

Published

Oct 17, 2026

This notebook walks through how to access the data and replicate the federal funds “hairy chart” in Python. The hairy chart plots the federal funds rate over time and, at each past date, the path the rate was then expected to take, drawn as a thin line, or “hair”.

We’ll load the federal funds target rate, then four sources of expected paths: the Treasury forward curve, options on SOFR futures, the FOMC’s median projection, and the Survey of Professional Forecasters. Each source is described, sourced, loaded and plotted on its own, followed by a simpler forecast a reader suggested, the 2-year Treasury yield. Then the four sources go on one plot, and a forecast evaluation scores them.

Each section summarizes how its data was downloaded and cleaned; the full pipeline, with a README on rebuilding the data, is in the repo’s scripts/ folder, and the cleaned files the notebook reads are in data/.

Setup

Packages, the color palette, plot theme, the per-source labels and captions, and helper functions that the tables and plots below reuse.

Code

import numpy as np
import pandas as pd
import altair as alt
from great_tables import GT, loc, style

# Charts render as static PNG images rather than interactive JavaScript
# widgets, which keeps the page light. scale_factor=2 doubles the pixel
# density so lines stay sharp on high-resolution screens.
alt.renderers.enable("png", scale_factor=2)
# Altair refuses data over 5,000 rows by default, as a guard against huge
# notebooks. The plots here draw up to about 32,000 points, and a PNG keeps
# none of that data, so the guard is switched off.
alt.data_transformers.disable_max_rows()

RATE = "#0B1B4A"
INK = "#1B3A6B"
INK_SOFT = "#5C7599"
GRID = "#C9D6E6"
BG = "#FFFFFF"

# Checked for separation under normal and color-blind vision. Python dicts keep
# their insertion order, so this one also fixes the order sources are listed in.
pal = {
    "treasury_forward": "#2E7DBF",
    "options_implied": "#C8102E",
    "sep_median": "#E08A1E",
    "spf_median": "#8C1A8C",
}

src_label = {
    "treasury_forward": "Treasury Forward Curve",
    "options_implied": "Options-Implied (SOFR)",
    "sep_median": "FOMC Median Projection",
    "spf_median": "Professional Forecasters",
}

# Named in the subtitle alongside the target rate.
src_phrase_subtitle = {
    "treasury_forward": "Treasury forward curves",
    "options_implied": "options-implied expected SOFR paths",
    "sep_median": "FOMC median projected rate paths",
    "spf_median": "professional forecasters' expected rate paths",
}

# Caption sources. Every plot shows the target rate, published by the Federal
# Reserve Board and retrieved through FRED, alongside its own hair source.
target_src = "Federal Reserve Board via the St. Louis Fed's FRED (target rate)"

src_caption = {
    "treasury_forward": "Federal Reserve Board, GSW nominal yield curve",
    "options_implied": "Federal Reserve Bank of Atlanta, Market Probability Tracker",
    "sep_median": "Federal Reserve Board, Summary of Economic Projections",
    "spf_median": "Federal Reserve Bank of Philadelphia, Survey of Professional Forecasters",
}


# The plot theme. Altair styles charts through a nested dict of settings;
# registering the function as a theme with enable=True applies it to every
# chart below.
@alt.theme.register("hairy", enable=True)
def theme_hairy():
    return {
        "config": {
            "background": BG,
            "view": {"stroke": None},  # no box around the plot area
            "title": {
                "anchor": "start",  # titles left-aligned
                "color": INK,
                "fontSize": 16,
                "subtitleColor": INK_SOFT,
                "subtitleFontSize": 12.5,
                "offset": 10,
            },
            # Every axis: no axis line, no ticks, soft labels.
            "axis": {
                "domain": False,
                "grid": False,
                "ticks": False,
                "labelColor": INK_SOFT,
                "labelFontSize": 11,
            },
            # Horizontal gridlines only, drawn from the y axis.
            "axisY": {"grid": True, "gridColor": GRID},
            # Year ticks on the x axis. Its title slot carries each plot's
            # sources line, styled as a small left-aligned caption.
            "axisX": {
                "ticks": True,
                "tickColor": INK_SOFT,
                "tickSize": 4,
                "titleAnchor": "start",
                "titleAlign": "left",
                "titleColor": INK_SOFT,
                "titleFontSize": 10,
                "titleFontWeight": "normal",
                "titlePadding": 12,
            },
            "legend": {"labelColor": INK, "labelFontSize": 11.5},
        }
    }


# Font family and size inherit from the page, so the tables match the body
# text wherever the notebook is embedded.
def gt_basic(d):
    return GT(d).tab_options(
        table_font_names="inherit",
        table_font_size="inherit",
        table_background_color=BG,
        column_labels_font_weight="bold",
        data_row_padding="5px",
        table_border_top_style="none",
    )


# One row per source: how many hairs, how many points each, how far apart the
# points are, and how much time a typical hair covers.
def hair_summary(d):
    # groupby("as_of") splits the table into one group per hair, and agg()
    # reduces each group to one row: name=function makes a column called name.
    per_hair = (
        d.sort_values(["as_of", "target_date"])
        .groupby("as_of")["target_date"]
        .agg(
            points="size",
            span=lambda s: (s.max() - s.min()).days,
            # Median days between consecutive points; NaN for a one-point hair.
            step=lambda s: s.diff().dt.days.median(),
        )
    )
    step = per_hair["step"].median()  # median() skips NaN by default
    points_per_hair = per_hair["points"].median()
    return pd.DataFrame({
        "Hairs": [len(per_hair)],
        "Points": [len(d)],
        # A median of counts is a float; shown as a whole number when it is one.
        "Points per hair": [int(points_per_hair) if points_per_hair.is_integer() else points_per_hair],
        "Point spacing": ["Monthly" if step < 45 else "Quarterly" if step < 120 else "Yearly"],
        "Typical span (months)": [round(per_hair["span"].median() / 30.44)],
    })


# Forecaster values are calendar-quarter averages, so those hairs are drawn as
# steps, each value held across its quarter. The other sources are joined by
# straight lines.
step_sources = ["spf_median"]


# Plot-ready hairs for one source. Each hair starts on the target line at its
# forecast date, reading the target one day later because FRED dates a change
# from the day it takes effect, so a forecast made at an FOMC meeting starts
# from that meeting's decision. Step sources get one row per quarter start,
# clipped to the forecast date, plus the last quarter's end. Plotting only;
# hairs.csv and the tables hold source points alone.
def hair_path(h, src):
    # The target rate indexed by date, so .map() can look values up by date.
    target_on = ffr.set_index("date")["target_mid"]

    # One anchor row per hair: x is the forecast date, y the target a day later.
    anchors = h[["as_of"]].drop_duplicates()
    anchors["target_date"] = anchors["as_of"]
    anchors["rate_pct"] = (anchors["as_of"] + pd.Timedelta(days=1)).map(target_on)
    anchors = anchors.dropna(subset=["rate_pct"])

    points = h[["as_of", "target_date", "rate_pct"]]
    if src in step_sources:
        # to_period("Q") turns each date into its calendar quarter; start_time
        # is the quarter's first day, and adding 1 steps to the next quarter.
        quarter = points["target_date"].dt.to_period("Q")
        starts = points.assign(target_date=np.maximum(quarter.dt.start_time, points["as_of"]))
        # The last point of each hair again, at the end of its quarter.
        last = points.loc[points.groupby("as_of")["target_date"].idxmax()]
        ends = last.assign(target_date=(last["target_date"].dt.to_period("Q") + 1).dt.start_time)
        points = pd.concat([starts, ends])

    # Anchors go first and the sort is stable, so where an anchor shares its
    # date with a hair's first point, the anchor still comes first. `order`
    # numbers the rows within each hair; Altair joins a line's points in that
    # order rather than re-sorting them by date.
    out = pd.concat([anchors, points], ignore_index=True)
    out = out.sort_values(["as_of", "target_date"], kind="stable")
    out["order"] = out.groupby("as_of").cumcount()
    return out


# One line per hair. Step sources hold each value flat until the next one
# ("step-after"); the rest are straight segments. clip=True cuts lines off at
# the plot edges instead of letting them run past the axes.
def hair_mark(src, data, **style):
    interpolate = "step-after" if src in step_sources else "linear"
    return alt.Chart(data).mark_line(interpolate=interpolate, clip=True, strokeCap="round", **style)


# Year axis: a label on round years, every `every` years.
def year_axis(start, end, every=5):
    first = int(np.ceil(start.year / every) * every)
    years = range(first, end.year + 1, every)
    return alt.Axis(values=[alt.DateTime(year=y, month=1, date=1) for y in years], format="%Y")


# A pandas Timestamp as an Altair DateTime, for axis limits.
def as_datetime(ts):
    return alt.DateTime(year=ts.year, month=ts.month, date=ts.day)


# Scale and axis settings shared by every layer of a plot: the date and rate
# limits, year labels, and a % on the rate axis. Each layer passes them on to
# its own x and y as **x and **y. The sources line goes in the x-axis title.
def axes(x0, x1, y0, y1, every=5, caption=None):
    x = dict(
        scale=alt.Scale(domain=[as_datetime(x0), as_datetime(x1)]),
        axis=year_axis(x0, x1, every),
        title=caption,
    )
    y = dict(
        scale=alt.Scale(domain=[y0, y1], nice=False),
        axis=alt.Axis(labelExpr="datum.label + '%'"),  # 4 becomes "4%"
        title=None,
    )
    return x, y


# The target rate, reused by every plot below: the target, or the midpoint of
# the target range, as a heavy line.
def rate_line(d, x, y):
    return alt.Chart(d).mark_line(color=RATE, strokeWidth=2, strokeCap="round", clip=True).encode(
        x=alt.X("date:T", **x), y=alt.Y("target_mid:Q", **y)
    )


def plot_source(src, pad_years=1):
    h = hairs[hairs["source"] == src]
    x0 = h["as_of"].min() - pd.Timedelta(days=365 * pad_years)
    x1 = h["target_date"].max() + pd.Timedelta(days=180)
    d = ffr[ffr["date"].between(x0, x1)]
    y_max = max(h["rate_pct"].max(), d["target_mid"].max()) * 1.08

    # Hair counts differ by a factor of twenty between sources, so a fixed alpha
    # leaves the SEP plot faint and the options plot a solid block.
    n = h["as_of"].nunique()
    a = max(0.08, min(0.55, 60 / n))

    # Labels every 5, 2 or 1 years depending on how much time the plot spans.
    span = (x1 - x0).days / 365.25
    every = 5 if span > 20 else 2 if span > 8 else 1

    x, y = axes(x0, x1, -0.3, y_max, every, caption=f"Sources: {src_caption[src]}; {target_src}.")
    lines = hair_mark(src, hair_path(h, src), color=pal[src], opacity=a, strokeWidth=1).encode(
        x=alt.X("target_date:T", **x),
        y=alt.Y("rate_pct:Q", **y),
        detail="as_of:T",  # one line per hair: rows sharing an as_of are joined
        order="order:Q",
    )
    # alt.layer() draws charts over each other on shared axes. Later layers
    # draw on top, so the target line goes last.
    return alt.layer(lines, rate_line(d, x, y)).properties(
        width=720,
        height=340,
        title=alt.Title(src_label[src], subtitle=f"Federal funds target rate and {src_phrase_subtitle[src]}"),
    )

Data

Federal Funds Target Rate

In a sense, this is the head from which all the hairs grow.

Downloaded from the FRED database, series DFEDTAR, DFEDTARL and DFEDTARU, by fetch_fred.py. The series are assembled by build_dataset.py, which splices the pre-2008 single target into the range columns so target_lo, target_hi and target_mid run continuously across the December 2008 regime change.

Source Detail: The FOMC Target (click to expand)

The rate the Federal Open Market Committee announces it is aiming for. A single number until December 2008, a quarter-point range from then on. The New York Fed: “The Federal Open Market Committee establishes the target rate, or range, for trading in the federal funds market.”

FRED DFEDTAR is the single target, 27 September 1982 to 15 December 2008. DFEDTARL and DFEDTARU are the lower and upper limits of the range from 16 December 2008, when the Committee cut to 0 to 0.25 percent and has used a range ever since. Verified at the join: DFEDTAR ends 2008-12-15 at 1.00, DFEDTARL/DFEDTARU begin 2008-12-16 at 0.00/0.25.

Values before 1994 are a reconstruction: the FOMC began announcing its target decisions in February 1994, and FRED’s series notes say the 1982 to 1993 values come from Thornton (2005), a Federal Reserve Bank of St. Louis working paper.

Citations.

https://fred.stlouisfed.org/series/DFEDTAR
https://fred.stlouisfed.org/series/DFEDTARU
https://fred.stlouisfed.org/series/DFEDTARL
Thornton, Daniel L. (2005), “A New Federal Funds Rate Target Series: September 27, 1982 - December 31, 1993”, Federal Reserve Bank of St. Louis
https://www.newyorkfed.org/markets/reference-rates/effr

Code

# read_csv() loads the file as a DataFrame, pandas' table type; parse_dates
# reads the date column as datetimes rather than text.
ffr = pd.read_csv("../data/fed_funds_rate.csv", parse_dates=["date"])

# One summary row. diff() is the change from the previous day, so counting the
# nonzero changes counts target changes; iloc[1:] skips the first day, which
# has no previous day. .pipe(gt_basic) passes the table on to gt_basic().
pd.DataFrame({
    "From": [ffr["date"].min().strftime("%b %Y")],
    "To": [ffr["date"].max().strftime("%b %Y")],
    "Days": [len(ffr)],
    "Target changes": [int((ffr["target_mid"].diff().iloc[1:] != 0).sum())],
    "Range era begins": [ffr.loc[ffr["target_lo"] != ffr["target_hi"], "date"].min().strftime("%d %b %Y")],
}).pipe(gt_basic)

From	To	Days	Target changes	Range era begins
Sep 1982	Oct 2026	16092	185	16 Dec 2008

Code

# A list of strings gives a caption of two lines.
x, y = axes(ffr["date"].min(), ffr["date"].max(), -0.3, 12, caption=[
    "Line is the target, or the midpoint of the target range since Dec 2008.",
    "Source: Federal Reserve Board via the St. Louis Fed's FRED (DFEDTAR, DFEDTARU, DFEDTARL).",
])
rate_line(ffr, x, y).properties(width=720, height=340, title="Federal Funds Target Rate")

Hairs

“Hairs” here are the predicted paths of the federal funds rate, from different sources. All four live in one tidy table, loaded once here and described one at a time below.

Code

hairs = pd.read_csv("../data/hairs.csv", parse_dates=["as_of", "target_date"])
# A categorical column holding the palette's order, so tables list sources in
# that order rather than alphabetically.
hairs["source"] = pd.Categorical(hairs["source"], categories=list(pal))

# One row per source. observed=True skips categories with no rows; each
# name=(column, function) pair in agg() makes one output column.
(
    hairs.groupby("source", observed=True)
    .agg(Hairs=("as_of", "nunique"), Points=("as_of", "size"), From=("as_of", "min"), To=("as_of", "max"))
    .reset_index()
    .assign(
        Source=lambda t: t["source"].astype(str).map(src_label),
        From=lambda t: t["From"].dt.strftime("%b %Y"),
        To=lambda t: t["To"].dt.strftime("%b %Y"),
    )[["Source", "Hairs", "Points", "From", "To"]]
    .pipe(gt_basic)
)

Source	Hairs	Points	From	To
Treasury Forward Curve	264	9504	Feb 1994	Sep 2026
Options-Implied (SOFR)	893	10217	Mar 2023	Oct 2026
FOMC Median Projection	44	155	Sep 2015	Sep 2026
Professional Forecasters	176	880	Nov 1982	Aug 2026

Treasury Forward Curve

Source Detail: GSW Nominal Yield Curve (click to expand)

What the Treasury bond market implies about future short rates, backed out daily by Federal Reserve Board staff from a smooth curve fitted across the whole maturity spectrum.

The Board describes the dataset as “smoothed yields on hypothetical Treasury securities that can be easily compared across maturities and over time, from 1961 to the present”. Its model “stipulates that the shape of the yield curve on any given date can be adequately captured by a set of six parameters”, estimated “by minimizing the discrepancy between the fitted Svensson yield curve and observed market yields”. Before 1980, “the Nelson-Siegel model—a model with fewer parameters—was used”.

On what goes into the fit: “only the coupon securities (Treasury notes and bonds) are used”; “on-the-run and first-off-the-run Treasury notes and bonds are not used”; and “Treasury bills and Treasury floating rate notes (FRNs) are not used”.

It is not an official statistic. The Board calls it “a staff research product and not an official statistical release”. A forward rate is the expected future short rate plus a term premium, so the hairs aren’t a pure expectations path: they can sit a little above or below it. And because bills are excluded and the curve is a six-parameter analytic function, the hairs are smooth by construction: they cannot show the discrete steps at FOMC meetings that market data pricing individual meetings shows.

Citations.

https://www.federalreserve.gov/data/nominal-yield-curve.htm
Gürkaynak, Sack and Wright (2006), FEDS 2006-28, https://www.federalreserve.gov/pubs/feds/2006/200628/200628pap.pdf
Published in Journal of Monetary Economics 54(8), 2007, 2291-2304

Downloaded from the Federal Reserve Board’s nominal yield curve data, as feds200628.csv. For each day, the file includes the six parameters of the fitted curve described in the source detail above. build_gsw_hairs.py plugs them into the formula for the instantaneous forward rate, the overnight rate the curve implies n years ahead:

\[ \begin{aligned} f(n) = {} & \beta_0 + \beta_1 e^{-n/\tau_1} \\ & + \beta_2 \tfrac{n}{\tau_1} e^{-n/\tau_1} + \beta_3 \tfrac{n}{\tau_2} e^{-n/\tau_2} \end{aligned} \]

The file also lists the forward rates themselves, but only for whole years ahead, so the script computes them for every month ahead; at whole years, the two agree. The results stay continuously compounded, which is close to how an overnight rate like fed funds is quoted.

build_dataset.py then samples the curve on FOMC meeting dates, taken from the Board’s meeting calendars by fetch_fomc_dates.py. build_long_hairs.py writes the same hairs out to 120 months, for the length comparison below; the Board publishes the curve out to 30 years.

Code

treasury_forward = hairs[hairs["source"] == "treasury_forward"]

# The same hairs uncapped, for the length comparison only.
treasury_long = pd.read_csv("../data/treasury_forward_120m.csv", parse_dates=["as_of", "target_date"])

hair_summary(treasury_forward).pipe(gt_basic)

Hairs	Points	Points per hair	Point spacing	Typical span (months)
264	9504	36	Monthly	35

What longer hairs for this series would look like, cut at 36, 60 and 120 months:

Code

cuts = [36, 60, 120]
x0, x1 = pd.Timestamp("1993-01-01"), pd.Timestamp("2036-12-31")
ffr_since_1993 = ffr[ffr["date"] >= x0]

# One panel per cut, stacked by vconcat() below. Only the bottom panel carries
# the sources line. {k / 12:g} prints 3.0 as "3".
panels = []
for k in cuts:
    caption = f"Sources: {src_caption['treasury_forward']}; {target_src}." if k == cuts[-1] else None
    x, y = axes(x0, x1, -0.3, 9.5, caption=caption)
    long_cut = hair_path(treasury_long[treasury_long["horizon_months"] <= k], "treasury_forward")
    lines = hair_mark("treasury_forward", long_cut, color=pal["treasury_forward"], opacity=0.25, strokeWidth=0.8).encode(
        x=alt.X("target_date:T", **x), y=alt.Y("rate_pct:Q", **y), detail="as_of:T", order="order:Q"
    )
    panels.append(
        alt.layer(lines, rate_line(ffr_since_1993, x, y)).properties(
            width=720, height=170, title=alt.Title(f"{k} months ({k / 12:g} years)", fontSize=12.5)
        )
    )

alt.vconcat(*panels, spacing=16).properties(
    title=alt.Title("Treasury Forward Curve, Three Hair Lengths", subtitle="Federal funds target rate and Treasury forward curves")
)

Hairs are cut at 36 months for legibility. I personally like the long fan rising out of the 2009 to 2015 floor: the market repeatedly pricing a liftoff that didn’t come until December 2015.

Code

plot_source("treasury_forward")

Options-Implied (SOFR)

Source Detail: Atlanta Fed Market Probability Tracker (click to expand)

What options traders are collectively betting, as a full probability distribution over where the overnight rate will average in a future three-month window.

From the data dictionary shipped inside the file: “The Market Probability Tracker uses Chicago Mercantile Exchange (CME) 3-month SOFR options to estimate the probability ditributions [sic] implied by their prices and specified reference window.” Their guide adds that it “transforms options on SOFR futures prices into probability distribution values”.

Two fields are used. reference_start is the “Start date of the 3-month window referenced by CME 3-month SOFR contracts”. Rate: mean is “The rate, in basis points, associated with the mean of the estimated distribution for a given reference window”.

The underlying rate is SOFR, published by the Federal Reserve Bank of New York: a broad measure of the cost of borrowing cash overnight collateralized by US Treasury securities. That makes it a secured rate, where the federal funds rate is unsecured lending between banks; the two are close but not identical. An option-implied distribution is risk-neutral, so it is what traders expect weighted by how much they care about being wrong each way, not literally what they expect. It starts in March 2023.

Each point comes from its own window’s option prices rather than from one fitted curve, but as a three-month average weighted across possible outcomes it does not resolve individual FOMC meetings.

Citations.

https://www.atlantafed.org/research-and-data/data/market-probability-tracker
https://www.atlantafed.org/research-and-data/data/market-probability-tracker/what-to-know
https://www.newyorkfed.org/markets/reference-rates/sofr
Estimation code published as mpt_source.zip on the tracker page

These hairs come from options on SOFR (Secured Overnight Financing Rate) futures. A SOFR futures contract settles on where SOFR averages over a future three-month window; options on that contract, across a range of strike prices, imply a probability distribution for the average. The Atlanta Fed estimates those distributions daily, and each hair is the mean of the distribution for each successive window, as of one day.

Downloaded from the Federal Reserve Bank of Atlanta’s Market Probability Tracker, as mpt_histdata.xlsx. build_dataset.py takes the DATA sheet, keeps field == "Rate: mean", divides by 100 to turn basis points into percent, and places each point at the midpoint of its reference window, which is reference_start plus 45 days.

Code

options_implied = hairs[hairs["source"] == "options_implied"]

hair_summary(options_implied).pipe(gt_basic)

Hairs	Points	Points per hair	Point spacing	Typical span (months)
893	10217	11	Quarterly	30

Code

plot_source("options_implied")

FOMC Median Projection

Source Detail: The Summary of Economic Projections (click to expand)

Four times a year each of the Federal Reserve’s senior policymakers writes down where they think the federal funds rate should be at the end of each of the next few years. This is the middle answer.

The table is titled “Economic projections of Federal Reserve Board members and Federal Reserve Bank presidents, under their individual assumptions of projected appropriate monetary policy”: all 7 Governors plus all 12 Reserve Bank presidents. That is wider than the Committee’s voting membership: the Board says the Committee “consists of twelve members”, namely “the seven members of the Board of Governors of the Federal Reserve System; the president of the Federal Reserve Bank of New York; and four of the remaining eleven Reserve Bank presidents, who serve one-year terms on a rotating basis”. Staff produce a separate forecast, the Tealbook, released only after five years.

From the table notes: “The projections for the federal funds rate are the value of the midpoint of the projected appropriate target range for the federal funds rate or the projected appropriate target level for the federal funds rate at the end of the specified calendar year or over the longer run.” And: “For each period, the median is the middle projection when the projections are arranged from lowest to highest.”

It is not a forecast. The row is labeled “Memo: Projected appropriate policy path”, and projections rest on each participant’s “assessment of appropriate monetary policy”. Normative, conditional on each person’s own outlook, and never voted on.

Only 44 of 59 releases give a median. The median column was introduced with the September 2015 SEP; before that the table shows a central tendency and range only, and the dots exist solely as a figure.

Citations.

https://www.federalreserve.gov/monetarypolicy/files/fomcprojtabl{YYYYMMDD}.pdf
https://www.federalreserve.gov/monetarypolicy/fomc.htm
https://fred.stlouisfed.org/series/FEDTARMD
https://www.federalreserve.gov/monetarypolicy/fomc_projectionsfaqs.htm

Downloaded from the Federal Reserve Board, one projection-table PDF per meeting, by fetch_forecasts.py. parse_sep.py reads the median row from each table; build_dataset.py dates each value to 31 December of its year and drops the longer-run value. Cross-checked against FRED.

Code

sep_median = hairs[hairs["source"] == "sep_median"]

hair_summary(sep_median).pipe(gt_basic)

Hairs	Points	Points per hair	Point spacing	Typical span (months)
44	155	4	Yearly	36

Four points per hair at most, one per projection year, dated to 31 December.

Code

plot_source("sep_median")

Professional Forecasters

Source Detail: Survey of Professional Forecasters (click to expand)

Every quarter the Philadelphia Fed asks a few dozen professional forecasters where the three-month Treasury bill rate will be over the next five quarters, and publishes the median.

The Philadelphia Fed describes it as “the oldest quarterly survey of macroeconomic forecasts in the United States. The survey began in 1968 and was conducted by the American Statistical Association and the National Bureau of Economic Research. The Federal Reserve Bank of Philadelphia took over the survey in 1990.” Individual responses are “kept confidential by using identification numbers”.

The survey asks about the bill rate, not the federal funds rate, but the two track each other closely: a three-month bill pays roughly what investors expect overnight rates to average over its three months. The bill normally sits a little below fed funds, so a forecast of the bill rate works as a rough forecast of the policy rate that runs slightly low.

Citations.

https://www.philadelphiafed.org/surveys-and-data/real-time-data-research/survey-of-professional-forecasters
Documentation PDF: variable definitions p.10, horizon numbering p.20, survey timing p.7

Downloaded from the Federal Reserve Bank of Philadelphia’s Survey of Professional Forecasters page, as median_tbill_level.xlsx, by fetch_forecasts.py. build_dataset.py drops TBILL1, the quarter before the survey, whose value forecasters already know, so horizons run from the current quarter (TBILL2) to four quarters ahead (TBILL6). It dates each hair at the response deadline, the middle of the quarter’s second month, and drops the five surveys from before September 1982, when the target series begins.

Code

spf_median = hairs[hairs["source"] == "spf_median"]

hair_summary(spf_median).pipe(gt_basic)

Hairs	Points	Points per hair	Point spacing	Typical span (months)
176	880	5	Quarterly	12

Five quarterly averages per hair, each drawn as a step across its quarter, covering about a year. The only one of the four reaching the 1980s.

Code

plot_source("spf_median")

2-Year Treasury Yield with Offset

I received a comment suggesting a simpler forecast than any of the four above: the 2-year Treasury yield, offset by five months, tracks the federal funds target. That is, take the day’s 2-year Treasury yield and use it as a prediction of the federal funds rate five months later.

The idea is plausible. A 2-year yield is roughly the overnight rate the market expects to average over the next two years, plus a term premium, so it tends to move before the Fed does. It’s also free, daily and goes back to 1976.

Below, we pull the data and set up these hairs. There’s a yield for every trading day, which would make a very hairy plot, so we draw one hair a week: from that day’s target rate to that day’s 2-year yield, five months out.

Source Detail: U.S. Treasury Securities at 2-Year Constant Maturity (click to expand)

The data is easily downloaded from the FRED database, series DGS2: the market yield on US Treasury securities at 2-year constant maturity, quoted on an investment basis, from the Federal Reserve Board’s H.15 release. Daily, in percent, from June 1976. The fetch_fred.py script downloads it with the other Treasury yields, and build_dataset.py writes it to data/treasury_2y.csv, dropping holidays, which have no quote.

Each trading day the Treasury fits a par yield curve to quotes on the most recently auctioned securities, and the 2-year constant maturity yield is read off that curve at exactly two years.

Each hair uses the yield on the last trading day of a week, not FRED’s weekly or monthly series (WGS2YR, GS2), which average the business days, so every hair starts on a day the yield was actually observed. It runs from the target that day to the yield, placed five months later. The hairs start with the 2-year series in June 1976 and stop at the last complete week.

Why five months? It was just a suggestion, not one chosen by testing, so in the end it is an arbitrary choice. The forecast evaluation below shows how other offsets do.

Citations.

Code

two_year = pd.read_csv("../data/treasury_2y.csv", parse_dates=["date"])
# DFF has a value for every calendar day, so a seven-day mean is one week.
# rolling(7).mean() averages each day with the six days before it.
effective = pd.read_csv("../data/fed_funds_effective.csv", parse_dates=["date"]).sort_values("date")
effective["week_mean"] = effective["rate_pct"].rolling(7).mean()

# This section's own color, caption and helpers, kept here so every other plot
# in the notebook renders as before.
two_year_color = "#1B9E77"
two_year_caption = (
    "Sources: Federal Reserve Board via the St. Louis Fed's FRED "
    "(2-year Treasury yield, DGS2; target rate; effective federal funds rate, DFF)."
)
offset_months = 5


# The same day k months after d, or the month's last day if it is shorter:
# pandas' DateOffset turns 31 January plus one month into 28 or 29 February.
def months_after(d, k):
    return d + pd.DateOffset(months=k)


# The rate on each date in d: the target, or before the target series begins,
# the effective rate's mean over the week to that date. map() looks each date
# up; fillna() fills the dates with no target from the effective rate.
target_on = ffr.set_index("date")["target_mid"]
week_mean_on = effective.set_index("date")["week_mean"]


def policy_rate(d):
    return d.map(target_on).fillna(d.map(week_mean_on))


# One hair a week, from the last trading day with a quote, in the same long
# format as hairs.csv. The latest week is left out: it may not be over.
# to_period("W-SUN") is the Monday-to-Sunday week each day falls in.
weekly = two_year.assign(week=two_year["date"].dt.to_period("W-SUN"))
weekly = weekly[weekly["week"] < weekly["week"].max()]
last_day_of_week = weekly.loc[weekly.groupby("week")["date"].idxmax()]
two_year_hairs = pd.DataFrame({
    "source": "two_year_offset",
    "as_of": last_day_of_week["date"],
    "target_date": months_after(last_day_of_week["date"], offset_months),
    "horizon_months": offset_months,
    "rate_pct": last_day_of_week["rate_pct"],
}).reset_index(drop=True)


# Each hair starts on the line at its forecast date, reading the line a day
# later as hair_path() does, then runs straight to its one point.
def two_year_path(h):
    anchors = pd.DataFrame({
        "as_of": h["as_of"],
        "target_date": h["as_of"],
        "rate_pct": policy_rate(h["as_of"] + pd.Timedelta(days=1)),
    })
    points = h[["as_of", "target_date", "rate_pct"]]
    return pd.concat([anchors, points]).dropna(subset=["rate_pct"])


# The line: the effective rate's weekly mean until the target series begins,
# then the target.
two_year_line = pd.concat([
    effective.loc[effective["date"] >= two_year_hairs["as_of"].min(), ["date", "week_mean"]]
    .rename(columns={"week_mean": "rate"})
    .assign(kind="effective"),
    ffr[["date", "target_mid"]].rename(columns={"target_mid": "rate"}).assign(kind="target"),
])

Code

x0, x1 = two_year_line["date"].min(), two_year_hairs["target_date"].max()
y1 = max(two_year_hairs["rate_pct"].max(), two_year_line["rate"].max()) * 1.04
x, y = axes(x0, x1, -0.3, y1, caption=[
    "The line is the target, or before September 1982 the effective rate, averaged over each week.",
    two_year_caption,
])

hair_lines = alt.Chart(two_year_path(two_year_hairs)).mark_line(
    color=two_year_color, opacity=0.6, strokeWidth=0.6, clip=True
).encode(x=alt.X("target_date:T", **x), y=alt.Y("rate_pct:Q", **y), detail="as_of:T")


# The effective rate is drawn thinner than the target.
def line_part(kind, width):
    d = two_year_line[two_year_line["kind"] == kind]
    return alt.Chart(d).mark_line(color=RATE, strokeWidth=width, strokeCap="round", clip=True).encode(
        x=alt.X("date:T", **x), y=alt.Y("rate:Q", **y)
    )


alt.layer(hair_lines, line_part("effective", 1), line_part("target", 2)).properties(
    width=720,
    height=400,
    title=alt.Title(
        "The 2-Year Treasury Yield as Hairs",
        subtitle="Federal funds rate, with a hair each week to that day's 2-year Treasury yield, five months out",
    ),
)

Three stretches up close: hikes then cuts, the long wait for liftoff, and cuts, COVID and the 2022 hikes.

Code

episodes = pd.DataFrame({
    "episode": [
        "2004 to 2007: hikes, then cuts",
        "2008 to 2015: waiting for liftoff",
        "2018 to 2024: cuts, COVID, the 2022 hikes",
    ],
    "from": pd.to_datetime(["2004-01-01", "2008-01-01", "2018-01-01"]),
    "to": pd.to_datetime(["2007-12-31", "2015-12-31", "2024-12-31"]),
})

all_paths = two_year_path(two_year_hairs)
# One rate scale for all three panels: up to the highest hair or rate in any.
in_any = [all_paths[all_paths["as_of"].between(e["from"], e["to"])]["rate_pct"].max() for _, e in episodes.iterrows()]
y1 = max(max(in_any), ffr.loc[ffr["date"] >= episodes["from"].min(), "target_mid"].max()) * 1.08

# One panel per stretch, each with its own dates, stacked by vconcat() below:
# hairs made in the stretch, and the target out to where the last of them ends.
panels = []
for i, e in episodes.iterrows():
    end = months_after(e["to"], offset_months)
    caption = two_year_caption if i == len(episodes) - 1 else None
    x, y = axes(e["from"], end, -0.3, y1, every=1, caption=caption)
    paths = all_paths[all_paths["as_of"].between(e["from"], e["to"])]
    lines = alt.Chart(paths).mark_line(color=two_year_color, opacity=0.6, strokeWidth=0.8, clip=True).encode(
        x=alt.X("target_date:T", **x), y=alt.Y("rate_pct:Q", **y), detail="as_of:T"
    )
    rate = ffr[ffr["date"].between(e["from"], end)]
    panels.append(
        alt.layer(lines, rate_line(rate, x, y)).properties(
            width=720, height=190, title=alt.Title(e["episode"], fontSize=12.5)
        )
    )

alt.vconcat(*panels, spacing=16).properties(
    title=alt.Title(
        "The 2-Year Treasury Yield, Three Episodes",
        subtitle="Federal funds target rate, with a hair each week to that day's 2-year Treasury yield, five months out",
    )
)

Forecast Evaluation for 2-Year Yields

How well does this forecast perform? Let’s check. For each hair whose end date has passed, that is, each hair whose accuracy we can check, we compare how far the target ended up from two forecasts: the 2-year Treasury yield, as a prediction for five months out, and no change, the target on the day the hair starts. Before September 1982 the effective rate stands in for the target. Errors are in percentage points, and “Hair closer” is the share of weeks where the hair’s error was the smaller of the two.

Code

two_year_check = two_year_hairs.assign(
    known=policy_rate(two_year_hairs["as_of"] + pd.Timedelta(days=1)),
    outcome=policy_rate(two_year_hairs["target_date"]),
).dropna(subset=["known", "outcome"])

# pd.cut() sorts each date into an era; right=False puts a boundary date into
# the era it starts.
two_year_check["Era"] = pd.cut(
    two_year_check["as_of"],
    bins=pd.to_datetime(["1976-01-01", "1982-09-27", "1994-01-01", "2008-01-01", "2016-01-01", "2100-01-01"]),
    labels=["1976 to 1982, effective rate", "1982 to 1993", "1994 to 2007", "2008 to 2015", "2016 on"],
    right=False,
)
two_year_check["yield_error"] = (two_year_check["rate_pct"] - two_year_check["outcome"]).abs()
two_year_check["flat_error"] = (two_year_check["known"] - two_year_check["outcome"]).abs()


def error_summary(d, era):
    return {
        "Era": era,
        "Weeks": len(d),
        "yield_abs": d["yield_error"].mean(),
        "flat_abs": d["flat_error"].mean(),
        "closer": (d["yield_error"] < d["flat_error"]).mean(),
    }


rows = [error_summary(d, era) for era, d in two_year_check.groupby("Era", observed=True)]
rows.append(error_summary(two_year_check, "All weeks"))

(
    pd.DataFrame(rows)
    .pipe(gt_basic)
    .tab_spanner("Mean absolute error", columns=["yield_abs", "flat_abs"])
    .cols_label(yield_abs="2-year yield", flat_abs="No change", closer="Hair closer")
    .fmt_number(columns=["yield_abs", "flat_abs"], decimals=2)
    .fmt_percent(columns="closer", decimals=0)
)

Era	Weeks	Mean absolute error		Hair closer
Era	Weeks	2-year yield	No change	Hair closer
1976 to 1982, effective rate	330	2.39	2.42	49%
1982 to 1993	588	1.13	0.73	32%
1994 to 2007	730	0.58	0.55	48%
2008 to 2015	418	0.50	0.15	5%
2016 on	541	0.35	0.45	50%
All weeks	2607	0.87	0.74	38%

Is five months the right offset? The same errors for every offset from 0 to 12 months. This is in-sample: choosing the best offset after looking at the results flatters it.

Code

offset_labels = {"yield": "2-year yield", "flat": "No change"}

rows = []
for k in range(13):
    d = two_year_hairs.assign(
        known=policy_rate(two_year_hairs["as_of"] + pd.Timedelta(days=1)),
        outcome=policy_rate(months_after(two_year_hairs["as_of"], k)),
    ).dropna(subset=["known", "outcome"])
    rows.append({"offset": k, "forecast": offset_labels["yield"], "mae": (d["rate_pct"] - d["outcome"]).abs().mean()})
    rows.append({"offset": k, "forecast": offset_labels["flat"], "mae": (d["known"] - d["outcome"]).abs().mean()})
offset_errors = pd.DataFrame(rows)

color = alt.Color(
    "forecast:N",
    title=None,
    scale=alt.Scale(domain=list(offset_labels.values()), range=[two_year_color, RATE]),
    legend=alt.Legend(orient="top-left", symbolType="stroke", symbolStrokeWidth=3, fillColor=BG, padding=6),
)
x = alt.X("offset:Q", axis=alt.Axis(values=list(range(13)), format="d", grid=False), title=["Offset (months)", two_year_caption])
y = alt.Y("mae:Q", scale=alt.Scale(domainMin=0), axis=alt.Axis(labelExpr="datum.label + ' pp'"), title=None)

lines = alt.Chart(offset_errors).mark_line(strokeWidth=1.6).encode(x=x, y=y, color=color)
points = alt.Chart(offset_errors).mark_point(filled=True, size=30).encode(x=x, y=y, color=color)
# A dashed rule at the five-month offset.
rule = alt.Chart(pd.DataFrame({"offset": [offset_months]})).mark_rule(color=GRID, strokeDash=[4, 3], strokeWidth=1.2).encode(x="offset:Q")

alt.layer(rule, lines, points).properties(
    width=720,
    height=280,
    title=alt.Title(
        "Mean Absolute Error by Offset, In-Sample",
        subtitle="How far the rate ended up from each forecast, by how many months ahead the hair is placed",
    ),
)

Other Sources

Other data on expected federal funds rates exist. The most direct are fed funds futures, exchange-traded contracts whose prices imply the market’s expected average federal funds rate for each coming month, and overnight index swap (OIS) curves, which do the same through swap contracts. Both price individual FOMC meetings, so their paths step rather than move smoothly, and they are the usual basis for this plot in market commentary. These series are not free (which is why they aren’t in this notebook; I couldn’t find a free copy anywhere): they require payment, through vendors such as Bloomberg, LSEG (Refinitiv), CME DataMine or Databento.

The sources above are free stand-ins for them.

The Very Hairy Plot

All four sources at once, against the target rate.

Code

x, y = axes(pd.Timestamp("1982-01-01"), pd.Timestamp("2029-12-31"), -0.3, 12, caption=[
    "Sources: Federal Reserve Board (GSW nominal yield curve; SEP; target rate, via the St. Louis Fed's FRED database);",
    "Federal Reserve Bank of Atlanta (Market Probability Tracker); Federal Reserve Bank of Philadelphia (SPF).",
])

# One color scale for all four hair layers, so they share a single legend,
# placed inside the plot at the top right.
color = alt.Color(
    "source:N",
    title=None,
    scale=alt.Scale(domain=list(src_label.values()), range=list(pal.values())),
    legend=alt.Legend(
        orient="top-right", symbolType="stroke", symbolStrokeWidth=3, symbolOpacity=1,
        fillColor=BG, padding=8,
    ),
)


def hair_layer(src, opacity, width):
    data = hair_path(hairs[hairs["source"] == src], src).assign(source=src_label[src])
    return hair_mark(src, data, opacity=opacity, strokeWidth=width).encode(
        x=alt.X("target_date:T", **x), y=alt.Y("rate_pct:Q", **y),
        color=color, detail="as_of:T", order="order:Q",
    )


# Layers draw in the order listed, so the target rate goes on top.
alt.layer(
    hair_layer("spf_median", 0.40, 1.0),
    hair_layer("treasury_forward", 0.25, 0.9),
    hair_layer("options_implied", 0.09, 0.8),
    hair_layer("sep_median", 0.55, 1.3),
    rate_line(ffr, x, y),
).properties(
    width=720,
    height=400,
    title=alt.Title("The Hairy Plot", subtitle="US federal funds target rate, with expected rate paths"),
)

Forecast Evaluation

How far off each source’s forecasts were, 6 to 36 months after they were made.

Code

# Horizons scored, in months. A month is 365/12 days, so 12 months is 365 days
# and 36 months is 1,095 days: exactly where the forecasters' and Treasury
# hairs end. round() rounds halves to even, as R does, so 6 months is 182 days.
horizons = [6, 12, 18, 24, 36]


def days_ahead(months):
    return round(months * 365 / 12)


# A cell is left blank with fewer forecasts than this, or when all of them
# were made within one horizon of each other: they then all overlap, and
# there is no way to tell how much the MAE would vary.
min_n = 8

# Every hair as it is drawn: the anchor on the target line, then the source's
# points (quarter steps for the forecasters). hair_path() puts an anchor before
# any point that shares its date, so keeping the last duplicate keeps the point.
paths = pd.concat(
    [hair_path(hairs[hairs["source"] == s], s).assign(source=s) for s in pal], ignore_index=True
).drop_duplicates(["source", "as_of", "target_date"], keep="last")


# Dates as whole day numbers, which numpy can interpolate between.
def day_number(d):
    return np.asarray(d, dtype="datetime64[D]").astype("int64")


# Read a hair at days `at`: straight lines between its points (np.interp), or
# the value held until the next point for step sources (searchsorted finds the
# last point on or before each day). NaN outside the hair, so a hair is never
# extended.
def read_hair(x, y, at, step):
    out = np.full(len(at), np.nan)
    inside = (at >= x[0]) & (at <= x[-1])
    if step:
        out[inside] = y[np.searchsorted(x, at[inside], side="right") - 1]
    else:
        out[inside] = np.interp(at[inside], x, y)
    return out


# The target on each of `days`: the last value on or before that day (the
# midpoint of the range since 2008).
ffr_days = day_number(ffr["date"])


def target_at(days):
    return ffr["target_mid"].to_numpy()[np.searchsorted(ffr_days, days, side="right") - 1]


# One row per hair and horizon: the forecast, and the target on that date.
# Only dates that have happened count.
last_day = day_number(ffr["date"].max())
ahead = np.array([days_ahead(h) for h in horizons])
rows = []
for (src, as_of), g in paths.groupby(["source", "as_of"]):
    about = day_number(as_of) + ahead
    forecast = read_hair(day_number(g["target_date"]), g["rate_pct"].to_numpy(), about, src in step_sources)
    rows.append(pd.DataFrame({"source": src, "as_of": as_of, "horizon": horizons, "about": about, "forecast": forecast}))
scored = pd.concat(rows, ignore_index=True)
scored = scored[scored["forecast"].notna() & (scored["about"] <= last_day)].copy()
scored["actual"] = target_at(scored["about"].to_numpy())
scored["abs_error"] = (scored["forecast"] - scored["actual"]).abs()

Code

# Newey-West standard error of a mean. Forecasts made close together cover
# mostly the same stretch of rates, so their errors move together and the
# plain sd / sqrt(N) would be far too small. Newey-West adds the
# autocovariances up to `lag`, with Bartlett weights 1 - l / (lag + 1).
# `@` is the dot product of two arrays.
def nw_se(x, lag):
    n = len(x)
    u = x - x.mean()
    lag = min(lag, n - 1)
    v = (u @ u) / n
    for l in range(1, lag + 1):
        v += 2 * (1 - l / (lag + 1)) * (u[l:] @ u[:-l]) / n
    return np.sqrt(v / n)


# The lag is how many forecasts a source makes within one horizon: two
# forecasts closer together than that overlap. Typical spacing is the median
# gap between a source's hairs (a day for options, a quarter for the SEP).
gap = {
    s: pd.Series(np.sort(hairs.loc[hairs["source"] == s, "as_of"].unique())).diff().dt.days.median()
    for s in pal
}


# MAE, its standard error and N for each source and horizon, within the extra
# grouping columns in `by`.
def summarise_errors(d, by=()):
    keys = ["source", *by, "horizon"]
    rows = []
    for key, g in d.sort_values("as_of", kind="stable").groupby(keys):
        row = dict(zip(keys, key))
        lag = int(np.ceil(days_ahead(row["horizon"]) / gap[row["source"]]))
        row.update(
            n=len(g),
            made_over=(g["as_of"].max() - g["as_of"].min()).days,
            mae=g["abs_error"].mean(),
            se=nw_se(g["abs_error"].to_numpy(), lag),
        )
        rows.append(row)
    return pd.DataFrame(rows)


# A cell: MAE, (SE), [N] on three lines, right-aligned in a monospaced font so
# the decimals line up. The trailing space on the MAE matches the closing
# parenthesis under it.
def eval_cell(r):
    if r["n"] < min_n or r["made_over"] < days_ahead(r["horizon"]):
        return ""
    return f'<span class="ev">{r["mae"]:.2f}&nbsp;<br>({r["se"]:.2f})<br>[{r["n"]}]</span>'


# Booktabs-style rules, the source name in the body font, the numbers in the
# monospaced one. fmt_passthrough(escape=False) lets the cells' HTML through.
def gt_eval(d, num_cols):
    return (
        gt_basic(d)
        .fmt_passthrough(columns=num_cols, escape=False)
        .cols_align("right", columns=num_cols)
        .tab_options(
            table_border_top_style="solid", table_border_top_width="2px", table_border_top_color=INK,
            heading_border_bottom_color=INK, column_labels_border_top_color=INK,
            column_labels_border_top_width="2px", column_labels_border_lr_color="transparent",
            table_border_bottom_width="2px", table_border_bottom_color=INK,
            column_labels_border_bottom_color=INK, column_labels_border_bottom_width="1px",
            table_body_hlines_color="transparent", table_body_border_bottom_color=INK,
            source_notes_font_size="13px", data_row_padding="8px",
        )
        .tab_style(style.text(whitespace="nowrap"), loc.body(columns="Source"))
        .opt_css(".ev { font-family: 'Courier Prime', ui-monospace, monospace; white-space: nowrap; line-height: 1.35; }")
    )


se_note = "(Newey-West standard error, which corrects for overlapping forecasts)"


# Cells for the forecasts made within each window of `windows` (columns
# `name`, `from`, `to`), one column per window. merge(how="cross") pairs every
# forecast with every window; pivot() turns the windows into columns.
def window_cells(d, windows):
    within = d.merge(windows, how="cross")
    within = within[within["as_of"].between(within["from"], within["to"])]
    s = summarise_errors(within, by=["name"])
    s["cell"] = s.apply(eval_cell, axis=1)
    return s.pivot(index="source", columns="name", values="cell").reset_index()


# One row per source, in the usual order, one column per window.
def window_table(cells, windows, spanner):
    d = pd.DataFrame({"source": list(pal)}).merge(cells, on="source", how="left")
    names = list(windows["name"])
    for nm in names:
        if nm not in d:
            d[nm] = ""
    d = d.assign(Source=d["source"].map(src_label))[["Source", *names]].fillna("")
    return gt_eval(d, names).tab_spanner(spanner, columns=names)

By Horizon

Code

s = summarise_errors(scored)
s["cell"] = s.apply(eval_cell, axis=1)
# One column per horizon; str() because table column names are text.
by_horizon = s.assign(horizon=s["horizon"].astype(str)).pivot(index="source", columns="horizon", values="cell")

cols = [str(h) for h in horizons]
counts = hairs.groupby("source", observed=True)["as_of"].agg(N="nunique", From=lambda d: d.min().strftime("%Y"))
counts.index = counts.index.astype(str)
table = counts.join(by_horizon).reset_index()
table["source"] = table["source"].astype(str)
for c in cols:
    if c not in table:
        table[c] = ""
table = table.assign(Source=table["source"].map(src_label))[["Source", "N", "From", *cols]]
table[cols] = table[cols].fillna("")

(
    gt_eval(table, cols)
    .cols_align("right", columns=["N", "From"])
    .tab_spanner("Months ahead", columns=cols)
    .tab_source_note(f"Each cell: mean absolute error in percentage points, {se_note}, [N forecasts scored]. N in the second column counts all of a source's hairs.")
    .tab_source_note("Error is the forecast at that horizon against the target rate, or the midpoint of the target range, on that date. Forecasts are read off each hair as drawn above, never past its last point.")
    .tab_source_note("None of the sources measures the target itself: Treasury forwards include a term premium, options price SOFR, and the forecasters predict the three-month bill, so part of each error is built in.")
    .tab_source_note(f"Blank: the source doesn't reach that far, or too few of its forecasts have come due (fewer than {min_n}, or all made within one horizon of each other).")
)

Source	N	From	Months ahead
Source	N	From	6	12	18	24	36
Treasury Forward Curve	264	1994	0.37 (0.05) [260]	0.85 (0.13) [256]	1.29 (0.20) [251]	1.64 (0.25) [247]	2.11 (0.32) [239]
Options-Implied (SOFR)	893	2023	0.26 (0.04) [768]	0.44 (0.08) [644]	0.48 (0.09) [518]
FOMC Median Projection	44	2015	0.32 (0.07) [42]	0.73 (0.24) [40]	1.16 (0.38) [38]	1.61 (0.51) [36]	2.15 (0.27) [17]
Professional Forecasters	176	1982	0.53 (0.06) [174]	1.02 (0.12) [172]
Each cell: mean absolute error in percentage points, (Newey-West standard error, which corrects for overlapping forecasts), [N forecasts scored]. N in the second column counts all of a source's hairs.
Error is the forecast at that horizon against the target rate, or the midpoint of the target range, on that date. Forecasts are read off each hair as drawn above, never past its last point.
None of the sources measures the target itself: Treasury forwards include a term premium, options price SOFR, and the forecasters predict the three-month bill, so part of each error is built in.
Blank: the source doesn't reach that far, or too few of its forecasts have come due (fewer than 8, or all made within one horizon of each other).

Three Stretches

The following table limits the 12-month MAE to forecasts made in three windows: 2004 to 2007 (hikes, then cuts), 2008 to 2015 (waiting for liftoff) and 2018 to 2024 (cuts, COVID and the 2022 hikes).

Code

# The three stretches plotted in the 2-year section, by the date each
# forecast was made. Scored 12 months ahead, the longest horizon all four
# sources reach.
stretches = pd.DataFrame({
    "name": ["2004 to 2007", "2008 to 2015", "2018 to 2024"],
    "from": pd.to_datetime(["2004-01-01", "2008-01-01", "2018-01-01"]),
    "to": pd.to_datetime(["2007-12-31", "2015-12-31", "2024-12-31"]),
})

(
    window_table(window_cells(scored[scored["horizon"] == 12], stretches), stretches, "12 months ahead, by year made")
    .tab_source_note(f"Each cell: mean absolute error in percentage points, {se_note}, [N forecasts scored].")
    .tab_source_note(f"Blank: fewer than {min_n} forecasts from the source in that stretch.")
)

Source	12 months ahead, by year made
Source	2004 to 2007	2008 to 2015	2018 to 2024
Treasury Forward Curve	1.11 (0.27) [32]	0.47 (0.18) [64]	1.03 (0.27) [57]
Options-Implied (SOFR)			0.50 (0.09) [444]
FOMC Median Projection			0.97 (0.28) [27]
Professional Forecasters	1.15 (0.37) [16]	0.55 (0.20) [32]	1.08 (0.31) [28]
Each cell: mean absolute error in percentage points, (Newey-West standard error, which corrects for overlapping forecasts), [N forecasts scored].
Blank: fewer than 8 forecasts from the source in that stretch.

Common Windows

Each source covers different years, so the rows above aren’t measured over the same dates. This table limits the 12-month MAE to forecasts made since the first FOMC median projection and since the first options-implied path, so more sources share the same dates.

Code

# Windows that start where a source starts, so each later column puts one
# more source on the same dates as the rest.
def first_made(src):
    return hairs.loc[hairs["source"] == src, "as_of"].min()


starts = [hairs["as_of"].min(), first_made("sep_median"), first_made("options_implied")]
common = pd.DataFrame({
    "name": ["All years", *[f"Since {d:%b %Y}" for d in starts[1:]]],
    "from": starts,
    "to": ffr["date"].max(),
})

common_note = (
    "All years: every forecast a source has. "
    f"{common['name'][1]}: made on or after {starts[1]:%d %b %Y}, when the FOMC median projection begins. "
    f"{common['name'][2]}: made on or after {starts[2]:%d %b %Y}, when the options-implied path begins; "
    "the only window all four sources share."
)

(
    window_table(window_cells(scored[scored["horizon"] == 12], common), common, "12 months ahead, by when made")
    .tab_source_note(f"Each cell: mean absolute error in percentage points, {se_note}, [N forecasts scored].")
    .tab_source_note(common_note + " Options-implied paths begin in March 2023, so their row is the same in every column.")
    .tab_source_note(f"Blank: the source has no forecasts in that window, or fewer than {min_n}.")
)

Source	12 months ahead, by when made
Source	All years	Since Sep 2015	Since Mar 2023
Treasury Forward Curve	0.85 (0.13) [256]	0.80 (0.22) [83]	0.41 (0.08) [21]
Options-Implied (SOFR)	0.44 (0.08) [644]	0.44 (0.08) [644]	0.44 (0.08) [644]
FOMC Median Projection	0.73 (0.24) [40]	0.73 (0.24) [40]	0.24 (0.03) [10]
Professional Forecasters	1.02 (0.12) [172]	0.82 (0.26) [40]	0.38 (0.08) [10]
Each cell: mean absolute error in percentage points, (Newey-West standard error, which corrects for overlapping forecasts), [N forecasts scored].
All years: every forecast a source has. Since Sep 2015: made on or after 17 Sep 2015, when the FOMC median projection begins. Since Mar 2023: made on or after 29 Mar 2023, when the options-implied path begins; the only window all four sources share. Options-implied paths begin in March 2023, so their row is the same in every column.
Blank: the source has no forecasts in that window, or fewer than 8.

Area Under the Curve

Rather than the miss on one date, the average gap between each hair and the target over the hair’s first 12 months: the area between the two lines, divided by 12 months. Same windows as above.

Code

# The gap between hair and target on every day from the forecast date to
# 365 days later, averaged: the area between the two lines over 12 months,
# divided by 12 months. Only hairs that reach 12 months, and whose 12 months
# have passed.
area_days = np.arange(days_ahead(12) + 1)
rows = []
for (src, as_of), g in paths.groupby(["source", "as_of"]):
    days = day_number(as_of) + area_days
    forecast = read_hair(day_number(g["target_date"]), g["rate_pct"].to_numpy(), days, src in step_sources)
    if np.isnan(forecast).any() or days[-1] > last_day:
        continue
    rows.append({"source": src, "as_of": as_of, "abs_error": np.abs(forecast - target_at(days)).mean()})
areas = pd.DataFrame(rows).assign(horizon=12)

(
    window_table(window_cells(areas, common), common, "Average gap over the first 12 months, by when made")
    .tab_source_note(f"Each cell: average absolute gap between hair and target over its first 12 months, in percentage points, {se_note}, [N forecasts].")
    .tab_source_note("The gap is read every day from the forecast date to 365 days later, along the hair as drawn above; forecasts whose 12 months haven't passed are left out.")
    .tab_source_note("Windows as in the table above. " + common_note)
    .tab_source_note(f"Blank: the source has no forecasts in that window, or fewer than {min_n}.")
)

Source	Average gap over the first 12 months, by when made
Source	All years	Since Sep 2015	Since Mar 2023
Treasury Forward Curve	0.40 (0.06) [256]	0.39 (0.09) [83]	0.27 (0.04) [21]
Options-Implied (SOFR)	0.25 (0.03) [644]	0.25 (0.03) [644]	0.25 (0.03) [644]
FOMC Median Projection	0.33 (0.08) [40]	0.33 (0.08) [40]	0.18 (0.02) [10]
Professional Forecasters	0.57 (0.07) [172]	0.40 (0.11) [40]	0.23 (0.02) [10]
Each cell: average absolute gap between hair and target over its first 12 months, in percentage points, (Newey-West standard error, which corrects for overlapping forecasts), [N forecasts].
The gap is read every day from the forecast date to 365 days later, along the hair as drawn above; forecasts whose 12 months haven't passed are left out.
Windows as in the table above. All years: every forecast a source has. Since Sep 2015: made on or after 17 Sep 2015, when the FOMC median projection begins. Since Mar 2023: made on or after 29 Mar 2023, when the options-implied path begins; the only window all four sources share.
Blank: the source has no forecasts in that window, or fewer than 8.

Notes

Where Does Each Hair Start?

Each hair starts on the target line at the date its forecast was made, since the rate is known then, and runs from there to the source’s first forecast; that first stretch is drawn, not forecast. A jump at the start of a hair reflects what the source measures (Treasury yields, SOFR or the three-month bill) rather than the target itself.

Session

Code

import sys
from importlib.metadata import version

print(f"Python {sys.version.split()[0]}")
for pkg in ["pandas", "numpy", "altair", "vl-convert-python", "great_tables"]:
    print(f"{pkg} {version(pkg)}")

Python 3.14.6
pandas 3.0.6
numpy 2.5.4
altair 6.3.0
vl-convert-python 1.9.0.post1
great_tables 1.1.0