Data Cleaning Notebook — Pavan Kalyan

In [1]

import pandas as pd
import numpy as np

In [2]

df=pd.read_csv("C:\\Users\\daroo\\Downloads\\Global YouTube Statistics.csv",encoding="latin1")

In [3]

print("Original Shape:",df.shape)
df.head()

Out

Original Shape: (995, 28)

	rank	Youtuber	subscribers	video views	category	Title	uploads	Country	Abbreviation	channel_type	...	subscribers_for_last_30_days	created_year	created_month	created_date	Gross tertiary education enrollment (%)	Population	Unemployment rate	Urban_population	Latitude	Longitude
0	1	T-Series	245000000	2.280000e+11	Music	T-Series	20082	India	IN	Music	...	2000000.0	2006.0	Mar	13.0	28.1	1.366418e+09	5.36	471031528.0	20.593684	78.962880
1	2	YouTube Movies	170000000	0.000000e+00	Film & Animation	youtubemovies	1	United States	US	Games	...	NaN	2006.0	Mar	5.0	88.2	3.282395e+08	14.70	270663028.0	37.090240	-95.712891
2	3	MrBeast	166000000	2.836884e+10	Entertainment	MrBeast	741	United States	US	Entertainment	...	8000000.0	2012.0	Feb	20.0	88.2	3.282395e+08	14.70	270663028.0	37.090240	-95.712891
3	4	Cocomelon - Nursery Rhymes	162000000	1.640000e+11	Education	Cocomelon - Nursery Rhymes	966	United States	US	Education	...	1000000.0	2006.0	Sep	1.0	88.2	3.282395e+08	14.70	270663028.0	37.090240	-95.712891
4	5	SET India	159000000	1.480000e+11	Shows	SET India	116536	India	IN	Entertainment	...	1000000.0	2006.0	Sep	20.0	28.1	1.366418e+09	5.36	471031528.0	20.593684	78.962880

5 rows × 28 columns

In [4]

df.isnull().sum()[df.isnull().sum()>0]

Out

category                                    46
Country                                    122
Abbreviation                               122
channel_type                                30
video_views_rank                             1
country_rank                               116
channel_type_rank                           33
video_views_for_the_last_30_days            56
subscribers_for_last_30_days               337
created_year                                 5
created_month                                5
created_date                                 5
Gross tertiary education enrollment (%)    123
Population                                 123
Unemployment rate                          123
Urban_population                           123
Latitude                                   123
Longitude                                  123
dtype: int64

In [5]

df.dtypes

Out

rank                                         int64
Youtuber                                    object
subscribers                                  int64
video views                                float64
category                                    object
Title                                       object
uploads                                      int64
Country                                     object
Abbreviation                                object
channel_type                                object
video_views_rank                           float64
country_rank                               float64
channel_type_rank                          float64
video_views_for_the_last_30_days           float64
lowest_monthly_earnings                    float64
highest_monthly_earnings                   float64
lowest_yearly_earnings                     float64
highest_yearly_earnings                    float64
subscribers_for_last_30_days               float64
created_year                               float64
created_month                               object
created_date                               float64
Gross tertiary education enrollment (%)    float64
Population                                 float64
Unemployment rate                          float64
Urban_population                           float64
Latitude                                   float64
Longitude                                  float64
dtype: object

In [6]

df=df[df["created_year"]!=1970]
print("Shape after removing 1970 rows:",df.shape)

Out

Shape after removing 1970 rows: (994, 28)

In [7]

df=df[df["uploads"]>0]
df=df[df["video views"]>0]
print("Shape After removing zero-upload/zero-view rows:",df.shape)

Out

Shape After removing zero-upload/zero-view rows: (948, 28)

In [8]

df["Country"]=df["Country"].fillna("Unknown")
df["Abbreviation"]=df["Abbreviation"].fillna("N/A")
df["channel_type"]=df["channel_type"].fillna("Unknown")
df["category"]=df["category"].fillna("Unknown")

In [9]

country_level_cols = [
    "Gross tertiary education enrollment (%)",
    "Population",
    "Unemployment rate",
    "Urban_population",
    "Latitude",
    "Longitude",
]
for col in country_level_cols:
    df[col]=df[col].fillna(df[col].median())

In [10]

rank_cols = ["video_views_rank", "country_rank", "channel_type_rank"]
for col in rank_cols:
    df[col]=df[col].fillna(df[col].median())
df["video_views_for_the_last_30_days"] = df["video_views_for_the_last_30_days"].fillna(0)
df["subscribers_for_last_30_days"] = df["subscribers_for_last_30_days"].fillna(0)

In [11]

df=df.dropna(subset=["created_year","created_month","created_date"])

In [12]

df["created_year"]=df["created_year"].astype(int)
df["created_date"]=df["created_date"].astype(int)

In [13]

df=df.drop_duplicates()
df=df.reset_index(drop=True)

In [14]

df.to_csv(r"C:\Users\daroo\Downloads\Global_YouTube_Statistics_Cleaned.csv", index=False)
print("Saved.")
print(df.head())

Out

Saved.
   Unnamed: 0  rank                    Youtuber  subscribers   video views  \
0           0     1                    T-Series    245000000  2.280000e+11   
1           2     3                     MrBeast    166000000  2.836884e+10   
2           3     4  Cocomelon - Nursery Rhymes    162000000  1.640000e+11   
3           4     5                   SET India    159000000  1.480000e+11   
4           6     7         ýýý Kids Diana Show    112000000  9.324704e+10   

         category                       Title  uploads        Country  \
0           Music                    T-Series    20082          India   
1   Entertainment                     MrBeast      741  United States   
2       Education  Cocomelon - Nursery Rhymes      966  United States   
3           Shows                   SET India   116536          India   
4  People & Blogs         ýýý Kids Diana Show     1111  United States   

  Abbreviation  ... subscribers_for_last_30_days  created_year  created_month  \
0           IN  ...                    2000000.0          2006            Mar   
1           US  ...                    8000000.0          2012            Feb   
2           US  ...                    1000000.0          2006            Sep   
3           IN  ...                    1000000.0          2006            Sep   
4           US  ...                          0.0          2015            May   

   created_date  Gross tertiary education enrollment (%)    Population  \
0            13                                     28.1  1.366418e+09   
1            20                                     88.2  3.282395e+08   
2             1                                     88.2  3.282395e+08   
3            20                                     28.1  1.366418e+09   
4            12                                     88.2  3.282395e+08   

   Unemployment rate  Urban_population   Latitude  Longitude  
0               5.36       471031528.0  20.593684  78.962880  
1              14.70       270663028.0  37.090240 -95.712891  
2              14.70       270663028.0  37.090240 -95.712891  
3               5.36       471031528.0  20.593684  78.962880  
4              14.70       270663028.0  37.090240 -95.712891  

[5 rows x 29 columns]

Global YouTube Statistics — Data Cleaning