Marvel Avengers Dataset

Alex Egg,

Avengers Data

You can also see this notebook rendered on github: https://github.com/eggie5/ipython-notebooks/blob/master/avengers/Avengers.ipynb

Life and Death of the Avengers

The Avengers are a well-known and widely loved team of superheroes in the Marvel universe that were introduced in the 1960’s in the original comic book series. They’ve since become popularized again through the recent Disney movies as part of the new Marvel Cinematic Universe.

The team at FiveThirtyEight wanted to dissect the deaths of the Avengers in the comics over the years. The writers were known to kill off and revive many of the superheroes so they were curious to know what data they could grab from the Marvel Wikia site, a fan-driven community site, to explore further. To learn how they collected their data, available on their Github repo, read the writeup they published on their site.

Exploring the Data

While the FiveThirtyEight team has done a wonderful job acquiring this data, the data still has some inconsistencies. Your mission, if you choose to accept it, is to clean up their dataset so it can be more useful for analysis in Pandas. First things first, let’s read our dataset into Padas as a DataFrame and preview the first 5 rows to get a better sense of our data.

import pandas as pd

avengers = pd.read_csv("avengers.csv")
avengers.head(5)
URL Name/Alias Appearances Current? Gender Probationary Introl Full/Reserve Avengers Intro Year Years since joining Honorary ... Return1 Death2 Return2 Death3 Return3 Death4 Return4 Death5 Return5 Notes
0 http://marvel.wikia.com/Henry_Pym_(Earth-616) Henry Jonathan "Hank" Pym 1269 YES MALE NaN Sep-63 1963 52 Full ... NO NaN NaN NaN NaN NaN NaN NaN NaN Merged with Ultron in Rage of Ultron Vol. 1. A...
1 http://marvel.wikia.com/Janet_van_Dyne_(Earth-... Janet van Dyne 1165 YES FEMALE NaN Sep-63 1963 52 Full ... YES NaN NaN NaN NaN NaN NaN NaN NaN Dies in Secret Invasion V1:I8. Actually was se...
2 http://marvel.wikia.com/Anthony_Stark_(Earth-616) Anthony Edward "Tony" Stark 3068 YES MALE NaN Sep-63 1963 52 Full ... YES NaN NaN NaN NaN NaN NaN NaN NaN Death: "Later while under the influence of Imm...
3 http://marvel.wikia.com/Robert_Bruce_Banner_(E... Robert Bruce Banner 2089 YES MALE NaN Sep-63 1963 52 Full ... YES NaN NaN NaN NaN NaN NaN NaN NaN Dies in Ghosts of the Future arc. However "he ...
4 http://marvel.wikia.com/Thor_Odinson_(Earth-616) Thor Odinson 2402 YES MALE NaN Sep-63 1963 52 Full ... YES YES NO NaN NaN NaN NaN NaN NaN Dies in Fear Itself brought back because that'...

5 rows × 21 columns

Filter out the bad years

Since the data was collected from a community site, where most of the contributions came from individual users, there’s room for errors to surface in the dataset. If you plot a histogram of the values in the Year column, which describe the year that Avenger was introduced, you’ll immediately notice some oddities. There are quite a few Avengers who look like they were introduced in 1900, which we know is a little fishy. The Avengers weren’t introduced in the comic series until the 1960’s!

import matplotlib.pyplot as plt
%matplotlib inline

true_avengers = pd.DataFrame()

avengers['Year'].hist()
<matplotlib.axes._subplots.AxesSubplot at 0x108cae890>

png

This is obviously a mistake in the data and you should remove all Avengers before 1960 from the DataFrame.

We only want to keep the Avengers who were introduced after 1960. Filter out all Avengers introduced before 1960 and store only the ones added in 1960 or later in `true_avengers`.
selector = avengers['Year'] > 1940
true_avengers = avengers[selector]
true_avengers['Year'].hist()
<matplotlib.axes._subplots.AxesSubplot at 0x10d2e5250>

png

Consolidating deaths

We are interested in the number of total deaths each character experienced and we’d like a field containing that distilled information. Right now, there are 5 fields (Death1 to Death5) that each contain a binary value representing if a superhero experienced that death or not. For example, a superhero can experience Death1, then Death2, etc. until they were no longer brought back to life by the writers.

We’d like to coalesce that information into just one field so we can do numerical analysis more easily.

Create a new column, Deaths, that contains the number of times each superhero died. The possible values for each death field are YES, NO, and the Pandas NaN value used to represent missing data. Keep all of the the original columns (including Death1 to Death5) and update true_avengers with the new Deaths column.

pd.options.mode.chained_assignment = None  # default='warn'
columns = ['Death1', 'Death2', 'Death3', 'Death4', 'Death5']
true_avengers[columns]


def clean_row(row):
    val = 0
    for column in columns:
        if(row[column] == "YES"):
            val += 1
        
    return val
            

death_column_vector = true_avengers.apply(lambda row: clean_row(row), axis=1)
true_avengers['Deaths']=death_column_vector
true_avengers.sort("Deaths", ascending=0)

URL Name/Alias Appearances Current? Gender Probationary Introl Full/Reserve Avengers Intro Year Years since joining Honorary ... Death2 Return2 Death3 Return3 Death4 Return4 Death5 Return5 Notes Deaths
32 http://marvel.wikia.com/Jocasta_(Earth-616)# Jocasta 141 YES FEMALE Jul-80 Nov-88 1988 27 Full ... YES YES YES YES YES YES YES YES From her article: Death1: "Defeated Ultron and... 5
29 http://marvel.wikia.com/Mar-Vell_(Earth-616)# Mar-Vell 254 NO MALE NaN Jul-78 1978 37 Full ... YES YES YES NO NaN NaN NaN NaN The bad penny of the Marvel universe. First ki... 3
81 http://marvel.wikia.com/Deathcry_(Earth-616)# NaN 50 NO FEMALE NaN Jul-93 1993 22 Honorary ... YES NO NaN NaN NaN NaN NaN NaN Reduced her to a pile of bones and organsby V... 2
18 http://marvel.wikia.com/Heather_Douglas_(Earth... Heather Douglas 332 NO FEMALE Jul-75 Sep-76 1976 39 Full ... YES YES NaN NaN NaN NaN NaN NaN Dies in Defenders_Vol_1_152. Later 'Obtained a... 2
51 http://marvel.wikia.com/Anthony_Druid_(Earth-6... Anthony Ludgate Druid 158 NO MALE NaN Apr-87 1987 28 Honorary ... YES YES NaN NaN NaN NaN NaN NaN Shot with a Breathing gun and corpse dumped in... 2
52 http://marvel.wikia.com/Marrina_Smallwood_(Ear... Marrina Smallwood 86 NO FEMALE NaN Dec-87 1987 28 Full ... YES YES NaN NaN NaN NaN NaN NaN Killed by Namor with the Ebony Blade. Returned... 2
53 http://marvel.wikia.com/Ravonna_Renslayer_(Ear... Ravonna Lexus Renslayer 41 NO FEMALE NaN May-88 1988 27 Full ... YES NO NaN NaN NaN NaN NaN NaN Killed in Avengers_Vol_1_24. Revived by the Gr... 2
55 http://marvel.wikia.com/Dennis_Dunphy_(Earth-6... Dennis Dunphy 70 NO MALE NaN Jan-88 1988 27 Full ... YES NO NaN NaN NaN NaN NaN NaN Died in Captain_America_348. Actually "Dunphy ... 2
73 http://marvel.wikia.com/Peter_Parker_(Earth-616)# Peter Benjamin Parker 4333 YES MALE NaN Apr-90 1990 25 Full ... YES YES NaN NaN NaN NaN NaN NaN Since joining the New Avengers: First death Ki... 2
78 http://marvel.wikia.com/Eric_Masterson_(Earth-... Eric Kevin Masterson 202 NO MALE NaN Jan-92 1992 23 Full ... YES NO NaN NaN NaN NaN NaN NaN Became posessed died to purge self of corrupti... 2
88 http://marvel.wikia.com/Jonathan_Hart_(Earth-6... Jonathan Hart 126 NO MALE NaN 1-Aug 2001 14 Full ... YES YES NaN NaN NaN NaN NaN NaN First died in a zero energy explosion in space... 2
105 http://marvel.wikia.com/Ares_(Earth-616)# Ares 236 NO MALE NaN 7-Mar 2007 8 Full ... YES NO NaN NaN NaN NaN NaN NaN Ripped in half by the Sentry during Seige. Rev... 2
20 http://marvel.wikia.com/Matthew_Hawk_(Earth-616)# Matthew Liebowitz (birth name) 197 NO MALE NaN Aug-75 1975 40 Full ... YES NO NaN NaN NaN NaN NaN NaN Died during Return_to_the_Old_West. Resurrecte... 2
94 http://marvel.wikia.com/Maya_Lopez_(Earth-616)# Maya Lopez 67 NO FEMALE NaN 5-Nov 2005 10 Full ... YES NO NaN NaN NaN NaN NaN NaN First death 'had been murdered in a fight with... 2
7 http://marvel.wikia.com/Clint_Barton_(Earth-616) Clinton Francis Barton 1456 YES MALE NaN May-65 1965 50 Full ... YES YES NaN NaN NaN NaN NaN NaN Dies in exploding Kree ship in Averngers Vol. ... 2
4 http://marvel.wikia.com/Thor_Odinson_(Earth-616) Thor Odinson 2402 YES MALE NaN Sep-63 1963 52 Full ... YES NO NaN NaN NaN NaN NaN NaN Dies in Fear Itself brought back because that'... 2
119 http://marvel.wikia.com/Thaddeus_Ross_(Earth-6... Thaddeus Ross 417 NO MALE NaN 11-Jun 2011 4 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Died in Circle of Four arc. Made a deal with M... 1
61 http://marvel.wikia.com/Dinah_Soar_(Earth-616)# NaN 22 NO FEMALE NaN Jul-89 1989 26 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Died in Great_Lakes_Avengers_Vol_1_1. Has not ... 1
154 http://marvel.wikia.com/Abyss_(Ex_Nihilo%27s)_... NaN 25 YES FEMALE NaN 13-Oct 2013 2 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Died in New_Avengers_Vol_3_32 1
70 http://marvel.wikia.com/Wade_Wilson_(Earth-616)# Wade Wilson 575 NO MALE NaN 7-Sep 2007 8 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Died during incursion. It's not ever particula... 1
68 http://marvel.wikia.com/Doug_Taggert_(Earth-616)# Doug Taggert 3 NO MALE NaN 5-May 2005 10 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Accidently killed by Zaran 1
67 http://marvel.wikia.com/Monkey_Joe_(Earth-616)# NaN 7 NO MALE NaN 5-May 2005 10 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Killed by Leather Boy 1
62 http://marvel.wikia.com/DeMarr_Davis_(Earth-616)# DeMarr Davis 31 YES MALE NaN Jul-89 1989 26 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Sacrificed self so that Mr. Immortal could sto... 1
59 http://marvel.wikia.com/Wendell_Vaughn_(Earth-... Wendell Elvis Vaughn 293 NO MALE NaN Oct-89 1989 26 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Killed by the Cosmic Assassin. Actually turned... 1
153 http://marvel.wikia.com/Ex_Nihilo_(Earth-616)# NaN 24 YES MALE NaN 13-Oct 2013 2 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Died in New_Avengers_Vol_3_32 1
56 http://marvel.wikia.com/Gilgamesh_(Earth-616)# NaN 61 NO MALE NaN Feb-89 1989 26 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Killed by Neut who was working for Immortus. R... 1
155 http://marvel.wikia.com/Nightmask_(Earth-616)# Adam 35 YES MALE NaN 13-Oct 2013 2 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Died in New_Avengers_Vol_3_32 1
54 http://marvel.wikia.com/Rita_DeMara_(Earth-616)# Rita DeMara 68 NO FEMALE NaN Nov-88 1988 27 Honorary ... NaN NaN NaN NaN NaN NaN NaN NaN Killed by Iron Man when he was controlled by I... 1
156 http://marvel.wikia.com/Kevin_Connor_(Earth-616)# Kevin Kale Connor 44 YES MALE NaN 13-Oct 2013 2 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Died in New_Avengers_Vol_3_32 1
164 http://marvel.wikia.com/Victor_Mancha_(Earth-6... Victor Mancha 75 YES MALE NaN 13-Sep 2013 2 Full ... NaN NaN NaN NaN NaN NaN NaN NaN Died in Avengers_A.I._Vol_1_4. Returned in Ave... 1
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
42 http://marvel.wikia.com/Marc_Spector_(Earth-616)# Marc Spector 402 NO MALE Sep-87 Jun-88 1988 27 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NA but he he did die and return to get his pow... 0
43 http://marvel.wikia.com/John_Walker_(Earth-616)# John F. Walker 352 NO MALE NaN May-89 1989 26 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
45 http://marvel.wikia.com/Miguel_Santos_(Earth-6... Miguel Santos 112 NO MALE NaN Apr-91 1991 24 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NA: Has not died since joing the Avengers but ... 0
46 http://marvel.wikia.com/Julia_Carpenter_(Earth... Julia Carpenter 218 NO FEMALE NaN Sep-92 1992 23 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
33 http://marvel.wikia.com/Greer_Nelson_(Earth-616)# Greer Grant Nelson 355 YES FEMALE NaN Sep-81 1981 34 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
31 http://marvel.wikia.com/Samuel_Wilson_(Earth-6... Samuel Thomas Wilson 576 YES MALE NaN Jun-79 1979 36 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
30 http://marvel.wikia.com/Carol_Danvers_(Earth-6... Carol Susan Jane Danvers 935 YES FEMALE NaN Apr-79 1979 36 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
84 http://marvel.wikia.com/Vance_Astrovik_(Earth-... Vance Astrovik 302 NO MALE NaN May-98 1998 17 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
108 http://marvel.wikia.com/Jessica_Jones_(Earth-6... Jessica Jones 205 YES FEMALE NaN 10-Aug 2010 5 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
107 http://marvel.wikia.com/Jessica_Drew_(Earth-616)# Jessica Miriam Drew 525 YES FEMALE NaN 8-Dec 2008 7 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
104 http://marvel.wikia.com/Stephen_Strange_(Earth... Doctor Stephen Vincent Strange 1324 NO MALE NaN 7-Feb 2007 8 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
103 http://marvel.wikia.com/Daniel_Rand_(Earth-616)# Daniel Thomas Rand K'ai 629 NO MALE NaN 7-Feb 2007 8 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
102 http://marvel.wikia.com/Thomas_Shepherd_(Earth... Thomas "Tommy" Shepherd 59 YES MALE NaN 6-Jun 2006 9 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
100 http://marvel.wikia.com/Katherine_Bishop_(Eart... Katherine "Kate" Bishop 132 YES FEMALE NaN 5-Jun 2005 10 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
23 http://marvel.wikia.com/Yondu_Udonta_(Earth-691)# Yondu Udonta 109 NO MALE NaN Feb-78 1978 37 Honorary ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
98 http://marvel.wikia.com/Dorrek_VIII_(Earth-616)# Dorrek VIII/Theodore "Teddy" Altman 110 YES MALE NaN 5-Apr 2005 10 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
97 http://marvel.wikia.com/William_Kaplan_(Earth-... William "Billy" Kaplan 123 YES MALE NaN 5-Apr 2005 10 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
96 http://marvel.wikia.com/Elijah_Bradley_(Earth-... Elijah Bradley 103 NO MALE NaN 5-Apr 2005 10 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
95 http://marvel.wikia.com/Nathaniel_Richards_(Ir... Nathaniel Richards 23 NO MALE NaN 5-Apr 2005 10 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NA is actually Kang but let's just not go there. 0
24 http://marvel.wikia.com/Martinex_T%27Naga_(Ear... Martinex T'Naga 100 NO MALE NaN Feb-78 1978 37 Honorary ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
25 http://marvel.wikia.com/Charlie-27_(Earth-691)# Charlie-27 132 NO MALE NaN Feb-78 1978 37 Honorary ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
26 http://marvel.wikia.com/Nicholette_Gold_(Earth... Nicholette Gold 108 NO FEMALE NaN Feb-78 1978 37 Honorary ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
27 http://marvel.wikia.com/Stakar_Ogord_(Earth-691)# Stakar 100 NO MALE NaN Feb-78 1978 37 Honorary ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
90 http://marvel.wikia.com/Luke_Cage_(Earth-616)# Carl Lucas 886 YES MALE NaN 5-Mar 2005 10 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
89 http://marvel.wikia.com/Kelsey_Leigh_(Earth-616)# Kelsey Leigh Shorr 24 NO FEMALE NaN 4-Jun 2004 11 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
28 http://marvel.wikia.com/Vance_Astro_(Earth-691)# Vance Astrovik 156 NO MALE NaN Feb-78 1978 37 Honorary ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
87 http://marvel.wikia.com/Maria_de_Guadalupe_San... Maria de Guadalupe Santiago 43 NO FEMALE NaN Jul-00 2000 15 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
86 http://marvel.wikia.com/Delroy_Garrett_Jr._(Ea... Delroy Garrett Jr. 101 NO MALE NaN Apr-00 2000 15 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
85 http://marvel.wikia.com/Angelica_Jones_(Earth-... Angelica Jones 330 NO FEMALE NaN May-98 1998 17 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0
172 http://marvel.wikia.com/Kaluu_(Earth-616)# Kaluu 35 YES MALE NaN 15-Jan 2015 0 Full ... NaN NaN NaN NaN NaN NaN NaN NaN NaN 0

159 rows × 22 columns

I sorted the output by the new Deaths column and it looks like some character “Jocasta” has died 5 times! Followed by Mar-Vell with 4 deaths.

Years since joining

For the final task, we want to know if the Years since joining field accurately reflects the Year column. If an Avenger was introduced in Year 1960, is the Years since joined value for that Avenger 55?

Calculate the number of rows where Years since joined is accurate. This challenge was created in 2015, so use that as the reference year. We want to know for how many rows Years since joined was correctly calculated as Year value subtracted from 2015.

joined_accuracy_count  = int()
correct_joined_years = true_avengers[true_avengers['Years since joining'] == (2015 - true_avengers['Year'])]
joined_accuracy_count = len(correct_joined_years)
joined_accuracy_count
159

Permalink: marvel-avengers-dataset

Tags:

Last edited by Alex Egg, 2015-09-30 05:15:20
View Revision History