Example of how to replace dataframe row missing (NaN) values using previous row values with pandas:
Create a dataframe with NaN values
Let's first create a dataframe with pandas with missing values:
import pandas as pd
import numpy as np
data = np.random.randint(100, size=(10,3))
df = pd.DataFrame(data=data,columns=['A','B','C'])
df.iloc[2,0:2] = np.nan
gives
A B C
0 16.0 4.0 90
1 78.0 16.0 1
2 NaN NaN 94
3 1.0 49.0 8
4 88.0 13.0 68
5 56.0 4.0 40
6 36.0 27.0 82
7 34.0 37.0 64
8 6.0 38.0 55
9 98.0 32.0 39
Replacing missing value using ffill
To fill dataframe row missing (NaN) values using previous row values with pandas, a solution is to use pandas.DataFrame.ffill:
df.ffill(inplace=True)
gives
A B C
0 16.0 4.0 90
1 78.0 16.0 1
2 78.0 16.0 94
3 1.0 49.0 8
4 88.0 13.0 68
5 56.0 4.0 40
6 36.0 27.0 82
7 34.0 37.0 64
8 6.0 38.0 55
9 98.0 32.0 39
Note: that missing values have been replaced by the values from the row just above.
Replacing multiple consequtive rows with missing values
Another example with multiple consequtive rows with missing values
import pandas as pd
import numpy as np
data = np.random.randint(100, size=(10,3))
df = pd.DataFrame(data=data,columns=['A','B','C'])
df.iloc[2,0:2] = np.nan
df.iloc[3,1:2] = np.nan
df.iloc[4,0:2] = np.nan
df.iloc[5,1:3] = np.nan
gives
A B C
0 83.0 0.0 50.0
1 27.0 29.0 18.0
2 NaN NaN 89.0
3 82.0 NaN 37.0
4 NaN NaN 76.0
5 42.0 NaN NaN
6 0.0 78.0 80.0
7 38.0 50.0 69.0
8 31.0 93.0 77.0
9 36.0 74.0 83.0
Then
df.ffill(inplace=True)
gives
A B C
0 83.0 0.0 50.0
1 27.0 29.0 18.0
2 27.0 29.0 89.0
3 82.0 29.0 37.0
4 82.0 29.0 76.0
5 42.0 29.0 76.0
6 0.0 78.0 80.0
7 38.0 50.0 69.0
8 31.0 93.0 77.0
9 36.0 74.0 83.0
Replacing missing value using with DataFrame.fillna()
Note: ffill() is synonym for DataFrame.fillna() with method='ffill'.
df.fillna(method='ffill')
also gives
A B C
0 16.0 4.0 90
1 78.0 16.0 1
2 78.0 16.0 94
3 1.0 49.0 8
4 88.0 13.0 68
5 56.0 4.0 40
6 36.0 27.0 82
7 34.0 37.0 64
8 6.0 38.0 55
9 98.0 32.0 39