DataScienceProgramminginPython-DataMunging
Background
BayAreaBikeSharecommencedit’spilotphaseofoperationintheSanFranciscobay
areainAugust2013withplanstoexpand.ItisthefirstbikesharingschemeinCalifornia.
Asitismeantforshorttrips,thebikesshouldbereturnedtoadockinthirtyminutesor
lessoranadditionalfeewouldbeincurredaccordingtothewebsite.Therearetwotypes
ofmemberships:customerandsubscriber.Asubscriberisanannualmembershipwhilea
customerisdefinedassomeoneusingeitherthetwenty-fourhourorthreedaypasses.
Currently(Sept2014),itcostsninedollarsfortwenty-fourhours,twenty-twodollarsfor
threedays,andeight-eightdollarsfortheyear.Overtimefeesarefourdollarsforanextra
thirtyminutesandsevendollarsforeachthirtyminutesafterthat.Dataonthefirstsix
monthsofoperationswerereleasedaspartofadatachallenge.Thedataincludedthree
filesfortriphistory,weatherinformation,anddockavailability.Themergeddatawasused
forthefollowinganalysis.
DataMungingandCarpentry
First,we’llreadinthedataandinspectthedatacolumnsanddatatypesandthinkabout
whatquestionswewanttoaskourdataandwhatthingsareweinterestedinlearningabout
thedata.Becuriousandempatheticinthinkingaboutwhatthevariousstakeholders
includingtheCity,thecustomers,andotherinterestedpeoplewouldbeinterestedin
gleaningbykeepingcivicfiscal,civic,andsocialgoalsinmind.Inadditiontothat,there
willbequiteabitofcleaninganddatacarpentryneededtogetthedataintoaformat
usefulforanalysis.
ThedatasetcomesfromthreecsvfilesfromtheBayAreaBikesharedatachallenge.We
mergedthedatainRaswestartedasimilaranalysistherebutreallywantedtouse
IPythonandthesuperbtimeseriesfunctionalityinPandas.
Optionally,onecancacheorlogthecodeinIPythonwiththefollowingtwocommands.
%load_extipycache
%logstart
Activatingauto-logging.Currentsessionstateplusfutureinputsaved.Filename:
ipython_log.py
Mode:rotate
Outputlogging:False
Rawinputlog:False
Timestamping:False
State:active
Importthelibrariesthatareneeded.
importnumpyasnp
importpandasaspd
