Understanding Wide and Long Data#

Tabular data can store the same values in two shapes: wide or long. hvPlot accepts both, but the shape of your data decides which options you reach for. This guide explains the two shapes, how hvPlot handles each of them, and when one is easier to work with than the other.

What wide and long mean#

In wide data, each series is stored in its own column. Here, each city’s temperatures are in a separate column:

import pandas as pd
import hvplot.pandas  # noqa

wide = pd.DataFrame({
    'month': ['Jan', 'Feb', 'Mar'],
    'Paris': [5, 6, 10],
    'Oslo': [-3, -2, 1],
})
wide

	month	Paris	Oslo
0	Jan	5	-3
1	Feb	6	-2
2	Mar	10	1

In long data (also called tidy data), each row is one observation. The values are stored in a single column, and another column says which series each value belongs to:

long = wide.melt(id_vars='month', var_name='city', value_name='temperature')
long

	month	city	temperature
0	Jan	Paris	5
1	Feb	Paris	6
2	Mar	Paris	10
3	Jan	Oslo	-3
4	Feb	Oslo	-2
5	Mar	Oslo	1

Both tables hold the same six temperatures. In the wide table, the city names are column names. In the long table, they are values of the city column. This difference matters for plotting: hvPlot options like by or groupby take a column name, so they can only group by information stored inside a column.

Wide data in hvPlot#

The stocks dataset is wide: one date column and one column of prices per company.

stocks = hvplot.sampledata.stocks('pandas')
stocks.head(3)

	date	Apple	Amazon	Google	Meta	Microsoft	Netflix
0	2019-01-01	1.00000	1.00000	1.00000	1.00000	1.00000	1.00000
1	2019-01-08	1.01399	0.99245	0.97782	1.05317	0.99990	1.05581
2	2019-01-15	1.06010	1.04093	1.02796	1.08685	1.05536	1.07535

When you don’t select y, hvPlot plots every numeric column, each as its own curve:

stocks.hvplot.line(x='date')

Behind the scenes, hvPlot turns the column names into a new categorical dimension, named Variable by default, and the values into a dimension named value. This is what you see in the legend title and the hover tooltip. Select a few columns with y=[...], and rename these two dimensions with group_label and value_label:

stocks.hvplot.line(
    x='date', y=['Apple', 'Google', 'Meta'],
    group_label='Company', value_label='Price',
)

With subplots=True, each column gets its own plot instead of being overlaid:

stocks.hvplot.line(
    x='date', y=['Apple', 'Google', 'Meta'],
    subplots=True, width=300,
)

Wide data is convenient when the series to compare are already columns. Its limit is that the series names only exist as column names, so they can’t be passed to options that expect a column, like groupby, row/col or color. For example, stocks.hvplot.line(x='date', groupby='Variable') raises an error, as there is no Variable column in the data.

Long data in hvPlot#

Reshaping the stocks data to long format gives one price column and one company column:

prices = stocks.melt(id_vars='date', var_name='company', value_name='price')
prices.head(3)

	date	company	price
0	2019-01-01	Apple	1.00000
1	2019-01-08	Apple	1.01399
2	2019-01-15	Apple	1.06010

The company names are now values in a column, so every grouping option accepts them. by overlays one curve per company, like the wide plot above:

prices.hvplot.line(x='date', y='price', by='company')

groupby shows one company at a time with a widget:

prices.hvplot.line(
    x='date', y='price',
    groupby='company', dynamic=False,
)

And col (or row) facets the plot into a grid:

prices.hvplot.line(x='date', y='price', col='company', width=200, height=200)

Long data can also hold several categorical columns at once. The penguins dataset is long: each row is one penguin, described by its species, island and sex along with its measurements.

penguins = hvplot.sampledata.penguins('pandas')
penguins.head(3)

	species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year
0	Adelie	Torgersen	39.1	18.7	181.0	3750.0	male	2007
1	Adelie	Torgersen	39.5	17.4	186.0	3800.0	female	2007
2	Adelie	Torgersen	40.3	18.0	195.0	3250.0	female	2007

Any of these columns can be used for grouping, and they can be combined, here coloring by species and adding widgets to select island and species:

penguins.hvplot.scatter(
    x='bill_length_mm', y='bill_depth_mm',
    groupby=['island', 'species'], color='sex',
    cmap={'male': 'blue', 'female': 'red', 'nan': 'green'},
    dynamic=False,
)

The Grouping and Coloring Options guide explains the differences between by, groupby and color.

When to use each#

Prefer wide data when:

the series to compare are already separate columns, like the stocks prices, and you want them overlaid or in subplots
you want a quick look at the distribution of several numeric columns, e.g. stocks.hvplot.box() draws one box per column

Prefer long data when:

you want widgets (groupby), a grid of plots (row/col) or a color per category (color)
the data has more than one categorical variable, like penguins with species, island and sex
you want to filter or aggregate by category with pandas before plotting

Many datasets are a mix of both: penguins is long with respect to its categories, but its four measurement columns are wide. Passing a list of them to y plots them side by side:

penguins.hvplot.box(
    y=['bill_length_mm', 'bill_depth_mm', 'flipper_length_mm'],
    group_label='Measurement',
)

Converting between wide and long#

pandas reshapes data in both directions. melt goes from wide to long, as done above with stocks.melt(id_vars='date', var_name='company', value_name='price'). pivot goes from long to wide:

prices_wide = prices.pivot(index='date', columns='company', values='price')
prices_wide.head(3)

company	Amazon	Apple	Google	Meta	Microsoft	Netflix
date
2019-01-01	1.00000	1.00000	1.00000	1.00000	1.00000	1.00000
2019-01-08	0.99245	1.01399	0.97782	1.05317	0.99990	1.05581
2019-01-15	1.04093	1.06010	1.02796	1.08685	1.05536	1.07535

After a pivot, date is the index, which hvPlot uses as the x-axis by default (see use_index), and the name of the columns (company) is picked up as the group label:

prices_wide.hvplot.line(y=['Apple', 'Google', 'Meta'])

See the pandas guide on reshaping and pivot tables for more options.

With polars, the same operations are called unpivot (wide to long) and pivot (long to wide). Polars has no index, so date stays a column and must be passed as x:

import hvplot.polars  # noqa

stocks_pl = hvplot.sampledata.stocks('polars')

prices_pl = stocks_pl.unpivot(index='date', variable_name='company', value_name='price')
prices_wide_pl = prices_pl.pivot(on='company', index='date', values='price')

prices_wide_pl.hvplot.line(x='date', y=['Apple', 'Google', 'Meta'])

Note

This guide is about tabular data. Gridded data like xarray objects is organized by named dimensions and coordinates rather than columns, and options like groupby accept these dimensions directly.

Next Steps

Understand the differences between by, groupby and color in Grouping and Coloring Options
See the data options reference for by, groupby, group_label, value_label and more