INTRODUCTION TO DATA SCIENCE - University Of Maryland

1 INTRODUCTION TO data SCIENCEJOHN P DICKERSONL ecture #6 2/14/2017 CMSC320 Tuesdays & Thursdays3:30pm 4:45pmANNOUNCEMENTSR egister on Piazza: 90 have registered already. I think we re good will release the first mini-project in less than one week. Please make sure Anaconda installed correctly! (See any of us if it didn t.) ELMS will be used for S LECTUREBy popular request .. Version control primer! Specifically, gitvia GitHuband GitLab Thanks: Mark Groves(Microsoft), IlanBiala& Aaron Perley(CMU), Sharif U., & the HJCB Senior Design Team!

2 And then a bit on keeping your data ..tidy IS VERSION CONTROL?4 DEVELOPMENT TOOLWhen working with a team, the need for a central repository is essential Need a system to allow versioning, and a way to acquire the latest edition of the code A system to track and manage bugs was also OF VERSION CONTROLBe able to search through revision history and retrieve previous versions of any file in a projectBe able to share changes with collaborators on a projectBe able to confidently make large changes to existing files6 NAMED FOLDERS APPROACHCan be hard to trackMemory-intensiveCan be slowHard to shareNo record of authorship7 LOCAL DATABASE OF VERSIONS APPROACHP rovides an abstraction over finding the right versions of files and

3 Replacing them in the projectRecords who changes what, but hard to parse thatCan t share with collaborators8 CENTRALIZED VERSION CONTROL SYSTEMSA central, trusted repository determines the order of commits ( versions of the project)Collaborators push changes (commits) to this new commits must be compatible with the most recent commit. If it isn t, somebody must merge it : SVN, CVS, Perforce9 Central RepositoryDeveloper A s local filesDeveloper D s local filesDeveloper C s local filesDeveloper B s local filesCommitCheckoutCheckoutCommitCommitC ommitCheckoutCheckoutDev A s RepoDev B s RepoDev C s RepoDev D s RepoCommitCommitCommitCommitPush/FetchPu sh/FetchPush/FetchPush/FetchPush/FetchPu sh/FetchCentralized Version Control SystemDistributed Version Control SystemDISTRIBUTED VERSION CONTROL SYSTEMS (DVCS) No central repository Every repository has every commit Examples.

4 Git, Mercurial10 WHAT IS GITGit is a version control systemDeveloped as a repository system for both local and remote changesAllows teammates to work simultaneously on a projectTracks each commit, allowing for a detailed documentation of the project along every stepAllows for advanced merging and branching operations11A SHORT HISTORY OF GITL inux kernel development1991-2002 Changes passed around as archived file2002-2005 Using a DVCS called BitKeeper2005 Relationship broke down between two communities (BitKeeperlicensing issues)12A SHORT HISTORY OF GITG oals: Speed Simple design Strong support for non-linear development (thousands of parallel branches) Fully distributed not a requirement, can be centralized Able to handle large projects like the Linux kernel efficiently (speed and data size)13A SHORT HISTORY OF GITP opularity.

5 Gitis now the most widely used source code management tool of professional software developers use Git(often through GitHub) as their primary source control system14[citation needed]GIT IN INDUSTRYC ompanies and projects currently using Git Google Android Facebook Microsoft Netflix Linux Ruby on Rails Gnome KDE Eclipse , not changes Apicture of what all your files look like at that moment If a file has not changed, store a referenceNearly every operation is local Browsing the history of project See changes between two versions16 WHY GIT IS BETTERGit tracks the content rather than the filesBranches are lightweight.

6 And merging is a simple processAllows for a more streamlined offline development processRepositories are smaller in size and are stored in a single .gitdirectoryAllows for advanced staging operations, and the use of stashing when working through troublesome sections17 WHAT ABOUT SVN?Linus TorvaldsSubversion has been the most pointless project ever started .. Subversion used to say CVS done right: with that slogan there is nowhere you can go. There is no way to do CVS right .. If you like using CVS, you should be in some kind of mental institution or somewhere VS {CVS, SVN.}

7 }Why you should care: Many places use legacy systems that will cause problems in the future be the change you believe in!Gitis muchfaster than SVN: Coded in C, which allows for a great amount of optimization Accomplishes much of the logic client side, thereby reducing time needed for communication Developed to work on the Linux kernel, so that large project manipulation is at the forefront of the benchmarks19 GITVS {CVS, SVN, ..}Speed benchmarks:Benchmarks performed by {CVS, SVN, ..}Git is significantly smaller than SVN All files are contained in a small decentralized.

8 Gitfile In the case of Mozilla s projects, a Git repository was 30 times smaller than an identical SVN repository Entire Linux kernel with 5 years of versioning contained in a single 1 GB .gitfile SVN carries two complete copies of each file, while Git maintains a simple and separate 100 bytes of data per file, noting changes and supporting operationsNice because you can (and do!) store the whole thing locally21 GITVS {CVS, SVN, ..}Gitis more securethan SVN All commits are uniquely hashed for both security and indexing purposes Commits can be authenticated through numerous means In the case of SSH commits, a key may be provided by both the client and server to guarantee authenticity and prevent against unauthorized access22 GIT VS {CVS, SVN.}

9 }Gitis decentralized: Each user contains an individual repository and can check commits against itself, allowing for detailed local revisioning Being decentralized allows for easy replication and deployment In this case, SVN relies on a single centralized repository and is unusable without23 GITVS {CVS, SVN, ..}Git is flexible: Due to it s decentralized nature, gitcommits can be stored locally, or committed through HTTP, SSH, FTP, or even by Email No need for a centralized repository Developed as a command line utility, which allows a large amount of features to be built and customized on top of it24 GITVS {CVS, SVN.

10 } data assurance: a checksum is performed on both upload and download to ensure sure that the file hasn t been IDs are generated upon each commit: Linked list style of commits Each commit is linked to the next, so that if something in the history was changed, each following commit will be rebranded to indicate the modification25 GITVS {CVS, SVN, ..}Branching: Git allows the usage of advanced branchingmechanisms and procedures Individual divisions of the code can be separated and developed separately within separate branches of the code Branches can allow for the separation of work between developers, or even for disposable experimentation Branching is a precursor and a component of the merging processWill give an example {CVS, SVN.