A common step in data analysis is to group the data by a variable and compute some summary statistics each subgroup of data. For example, one might be interested in mean, median values, or total sum per group. In this post, we will see an example of how to use groupby() function in Pandas to group a dataframe into multiple smaller dataframes and compute total/sum on another variable.
Let us load the libraries we need.
import pandas as pd import numpy as np
We will use gapminder dataset to learn groupby() and sum() functions to summarise data at a group level.
p2data = "https://raw.githubusercontent.com/cmdlinetips/data/master/gapminder-FiveYearData.csv" gapminder=pd.read_csv(p2data) gapminder.head()
Let us first subset the data for the sake of simplicity. Here we filter data for year values 2007 using Pandas filter() function.
df= gapminder.query("year==2007")
df.head()
With the data corresponding to the year 2007, let us compute total population in each continent. In order to do that, we first need to use groupby() to group the data corresponding to each continent.
df.groupby(["continent"]) <pandas.core.groupby.generic.DataFrameGroupBy object at 0x1a19c9c850>
From the grouped object, let us select our variable of interest. Since we are interested in computing total population, we select “pop”
df.groupby(["continent"])['pop'] <pandas.core.groupby.generic.SeriesGroupBy object at 0x1a19cc1590>
And chain it with sum() function in Pandas that computes the total population for each continent.
df.groupby(["continent"])['pop'].sum()
Here we have results as Pandas Series with total population for each continent computed by groupby() and sum().
continent Africa 9.295397e+08 Americas 8.988712e+08 Asia 3.811954e+09 Europe 5.860985e+08 Oceania 2.454995e+07 Name: pop, dtype: float64
This post is part of the series on Pandas 101, a tutorial covering tips and tricks on using Pandas for data munging and analysis.
