How to fill dataframe row missing (NaN) values using previous row values with pandas ?


Example of how to replace dataframe row missing (NaN) values using previous row values with pandas:

Create a dataframe with NaN values

Let's first create a dataframe with pandas with missing values:

import pandas as pd
import numpy as np

data = np.random.randint(100, size=(10,3))

df = pd.DataFrame(data=data,columns=['A','B','C'])

df.iloc[2,0:2] = np.nan

gives

      A     B   C
0  16.0   4.0  90
1  78.0  16.0   1
2   NaN   NaN  94
3   1.0  49.0   8
4  88.0  13.0  68
5  56.0   4.0  40
6  36.0  27.0  82
7  34.0  37.0  64
8   6.0  38.0  55
9  98.0  32.0  39

Replacing missing value using ffill

To fill dataframe row missing (NaN) values using previous row values with pandas, a solution is to use pandas.DataFrame.ffill:

df.ffill(inplace=True)

gives

      A     B   C
0  16.0   4.0  90
1  78.0  16.0   1
2  78.0  16.0  94
3   1.0  49.0   8
4  88.0  13.0  68
5  56.0   4.0  40
6  36.0  27.0  82
7  34.0  37.0  64
8   6.0  38.0  55
9  98.0  32.0  39

Note: that missing values have been replaced by the values from the row just above.

Replacing multiple consequtive rows with missing values

Another example with multiple consequtive rows with missing values

import pandas as pd
import numpy as np

data = np.random.randint(100, size=(10,3))

df = pd.DataFrame(data=data,columns=['A','B','C'])

df.iloc[2,0:2] = np.nan
df.iloc[3,1:2] = np.nan
df.iloc[4,0:2] = np.nan
df.iloc[5,1:3] = np.nan

gives

      A     B     C
0  83.0   0.0  50.0
1  27.0  29.0  18.0
2   NaN   NaN  89.0
3  82.0   NaN  37.0
4   NaN   NaN  76.0
5  42.0   NaN   NaN
6   0.0  78.0  80.0
7  38.0  50.0  69.0
8  31.0  93.0  77.0
9  36.0  74.0  83.0

Then

df.ffill(inplace=True)

gives

      A     B     C
0  83.0   0.0  50.0
1  27.0  29.0  18.0
2  27.0  29.0  89.0
3  82.0  29.0  37.0
4  82.0  29.0  76.0
5  42.0  29.0  76.0
6   0.0  78.0  80.0
7  38.0  50.0  69.0
8  31.0  93.0  77.0
9  36.0  74.0  83.0

Replacing missing value using with DataFrame.fillna()

Note: ffill() is synonym for DataFrame.fillna() with method='ffill'.

df.fillna(method='ffill')

also gives

      A     B   C
0  16.0   4.0  90
1  78.0  16.0   1
2  78.0  16.0  94
3   1.0  49.0   8
4  88.0  13.0  68
5  56.0   4.0  40
6  36.0  27.0  82
7  34.0  37.0  64
8   6.0  38.0  55
9  98.0  32.0  39

References