Pandas DataFrame Manipulation

DataFrame manipulation in Pandas involves editing and modifying existing DataFrames. Some common DataFrame manipulation operations are:

• Adding rows/columns
• Removing rows/columns
• Renaming rows/columns

Add a New Column to a Pandas DataFrame

We can add a new column to an existing Pandas DataFrame by simply declaring a new list as a column. For example,

import pandas as pd

# define a dictionary containing student data
data = {'Name': ['John', 'Emma', 'Michael', 'Sophia'],
        'Height': [5.5, 6.0, 5.8, 5.3],
        'Qualification': ['BSc', 'BBA', 'MBA', 'BSc']}

# convert the dictionary into a DataFrame
df = pd.DataFrame(data)

# declare a new list
address = ['New York', 'London', 'Sydney', 'Toronto']

# assign the list as a column
df['Address'] = address

print(df)

Output

      Name  Height Qualification   Address
0     John     5.5           BSc  New York
1     Emma     6.0           BBA    London
2  Michael     5.8           MBA    Sydney
3   Sophia     5.3           BSc   Toronto

In this example, we assign the list address to the Address column in the DataFrame.

Add a New Row to a Pandas DataFrame

Adding rows to a DataFrame is not quite as straightforward as adding columns in Pandas. We use the .loc property to add a new row to a Pandas DataFrame.

For example,

import pandas as pd

# define a dictionary containing student data
data = {'Name': ['John', 'Emma', 'Michael', 'Sophia'],
        'Height': [5.5, 6.0, 5.8, 5.3],
        'Qualification': ['BSc', 'BBA', 'MBA', 'BSc']}

# convert the dictionary into a DataFrame
df = pd.DataFrame(data)

print("Original DataFrame:")
print(df)
print()

# add a new row
df.loc[len(df.index)] = ['Amy', 5.2, 'BIT'] 

print("Modified DataFrame:")
print(df)

Output

Original DataFrame:
      Name  Height Qualification
0     John    5.5           BSc
1     Emma    6.0           BBA
2  Michael    5.8           MBA
3   Sophia    5.3           BSc

Modified DataFrame:
      Name  Height Qualification
0     John    5.5           BSc
1     Emma    6.0           BBA
2  Michael    5.8           MBA
3   Sophia    5.3           BSc
4      Amy    5.2           BIT

In this example, we added a row ['Amy', 5.2, 'BIT'] to the df DataFrame.

Here,

• len(df.index): returns the number of rows in df
• df.loc[...]: accesses the row with index value enclosed by the square brackets

To learn more about .loc, please visit Pandas Indexing and Slicing.

Remove Rows/Columns from a Pandas DataFrame

We can use drop() to delete rows and columns from a DataFrame.

Example: Delete Rows

import pandas as pd

# create a sample DataFrame
data = {'Name': ['Alice', 'Bob', 'Charlie', 'David', 'Felipe', 'Rita'],
        'Age': [25, 30, 35, 40, 22, 29],
        'City': ['New York', 'London', 'Paris', 'Tokyo', 'Bogota', 'Banglore']}
df = pd.DataFrame(data)

# display the original DataFrame
print("Original DataFrame:")
print(df)
print()

# delete row with index 4
df.drop(4, axis=0, inplace=True)

# delete row with index 5
df.drop(index=5, inplace=True)

# delete rows with index 1 and 3
df.drop([1, 3], axis=0, inplace=True)

# display the modified DataFrame after deleting rows
print("Modified DataFrame:")
print(df)

Output

Original DataFrame:
      Name  Age      City
0    Alice   25  New York
1      Bob   30    London
2  Charlie   35     Paris
3    David   40     Tokyo

Modified DataFrame:
      Name  Age      City
0    Alice   25  New York
2  Charlie   35     Paris

In this example, we deleted single rows using the labels=4 and index=5 parameters. We also deleted multiple rows with labels=[1,3] argument.

Here,

• axis=0: indicates that rows are to be deleted
• inplace=True: indicates that the changes are to be made in the original DataFrame

Example: Delete columns

import pandas as pd

# create a sample DataFrame
data = {'Name': ['Alice', 'Bob', 'Charlie', 'David'],
        'Age': [25, 30, 35, 40],
        'City': ['New York', 'London', 'Paris', 'Tokyo'],
        'Height': ['165', '178', '185', '171'],
        'Profession': ['Engineer', 'Entrepreneur', 'Unemployed', 'Actor'],
        'Marital Status': ['Single', 'Married', 'Divorced', 'Engaged']}
df = pd.DataFrame(data)

# display the original DataFrame
print("Original DataFrame:")
print(df)
print()

# delete age column
df.drop('Age', axis=1, inplace=True)

# delete marital status column
df.drop(columns='Marital Status', inplace=True)

# delete height and profession columns
df.drop(['Height', 'Profession'], axis=1, inplace=True)

# display the modified DataFrame after deleting rows
print("Modified DataFrame:")
print(df)

Output

Original DataFrame:
      Name  Age      City   Height    Profession   Marital Status
0    Alice  25  New York    165      Engineer         Single
1      Bob  30    London    178  Entrepreneur        Married
2  Charlie  35     Paris    185    Unemployed       Divorced
3    David  40     Tokyo    171         Actor        Engaged

Modified DataFrame:
      Name      City
0    Alice  New York
1      Bob    London
2  Charlie     Paris
3    David     Tokyo

In this example, we deleted single columns using the labels='Age' and columns='Marital Status' parameters. We also deleted multiple columns with labels=['Height', 'Profession'] argument.

Here,

• axis=1: indicates that columns are to be deleted
• inplace=True: indicates that the changes are to be made in the original DataFrame

Rename Labels in a DataFrame

We can rename columns in a Pandas DataFrame using the rename() function.

Example: Rename Columns

import pandas as pd

# create a sample DataFrame
data = {'Name': ['Alice', 'Bob', 'Charlie', 'David'],
        'Age': [25, 30, 35, 40],
        'City': ['New York', 'London', 'Paris', 'Tokyo']}
df = pd.DataFrame(data)

# display the original DataFrame
print("Original DataFrame:")
print(df)
print()

# rename column 'Name' to 'First_Name'
df.rename(columns= {'Name': 'First_Name'}, inplace=True)

# rename columns 'Age' and 'City'
df.rename(mapper= {'Age': 'Number', 'City':'Address'}, axis=1, inplace=True)

# display the DataFrame after renaming column
print("Modified DataFrame:")
print(df)

Output

Original DataFrame:
      Name  Age      City
0    Alice  25  New York
1      Bob  30    London
2  Charlie  35     Paris
3    David  40     Tokyo

Modified DataFrame:
  First_Name  Number   Address
0      Alice     25  New York
1        Bob     30    London
2    Charlie     35     Paris
3      David     40     Tokyo

In this example, we renamed a single column using the columns={'Name': 'First_Name'} parameter. We also renamed multiple columns with mapper={'Age': 'Number', 'City':'Address'} argument.

Here,

• axis=1: indicates that columns are to be renamed
• inplace=True: indicates that the changes are to be made in the original DataFrame

Example: Rename Row Labels

import pandas as pd

# create a sample DataFrame
data = {'Name': ['Alice', 'Bob', 'Charlie', 'David'],
        'Age': [25, 30, 35, 40],
        'City': ['New York', 'London', 'Paris', 'Tokyo']}
df = pd.DataFrame(data)

# display the original DataFrame
print("Original DataFrame:")
print(df)
print()

# rename column one index label
df.rename(index={0: 7}, inplace=True)

# rename columns multiple index labels
df.rename(mapper={1: 10, 2: 100}, axis=0, inplace=True)

# display the DataFrame after renaming column
print("Modified DataFrame:")
print(df)

Output

Original DataFrame:
      Name  Age      City
0    Alice   25  New York
1      Bob   30    London
2  Charlie   35     Paris
3    David   40     Tokyo

Modified DataFrame:
        Name  Age      City
7      Alice   25  New York
10       Bob   30    London
100  Charlie   35     Paris
3      David   40     Tokyo

In this example, we renamed a single row using the index={0: 7} parameter. We also renamed multiple rows with mapper={1: 10, 2: 100} argument.

Here,

• axis=0: indicates that rows are to be renamed
• inplace=True: indicates that the changes are to be made in the original DataFrame