- Start day: 2023.01.08
- Session 1 - Welcome - Version Control - Journal - Slack
- Assignment 1
- Session 2 - Create GitHub Pages using the GitHub UI, Markdown
- Assignment 2
- Session 3 - git client
- Assignment 3
- Session 4 - Upload your own project - testing
- Assignment 4
- Assignment 5
- Assignment 6
- Session 7
- Assignment 7
- Session 8
Session 1 - Welcome - Version Control - Journal - Slack
- overview of the course
- Static analysis
- a little about myself
- Self employed
- Introducing testing, CI etc.
- If you'd like to send me an email reply to the one I sent you. Keep the subject line. Remove the irrelevant content. Without this it is very difficult for me to associate all the emails with the different courses I teach.
- Will be in some public GitHub or GitLab repositories
- At the end of each assignment you'll write a report - a blog post / journal entry.
- You will add it to your personal JSON file and send a Pull-Request with the change. (We'll learn these soon)
- The more you participate, the more you learn in this course.
- The more effort you put in this course, the more you will gain.
- Ask questions!
- Try to help others! The more you help others the more you will learn.
- Grades: (if relevant) are based on the work done during the course. There is no end-project or exam at the end.
- overview of the course
GitHub: process of contributing to an Open Source project using the GitHub web site. Editing and sending a Pull-Request. Use a the
cm-demouser to make a change in the README of this repository and then to add the json file. Show how the CI fails when we add an incorrectly formatted file.
What is JSON?
Show the Git repository of the project and the web site generated from it.
After the video recording also mentioned https://kantoniko.com/
- You will have to publish a journal of your process. You can use any blogging platform, but let me suggest a few:
- Create an account on the blogging platform you selected. (if you already have one, you can use that)
- Create an account on GitHub (if you already have one, use that)
- Create an account on GitLab (if you already have one, use that)
- Add a picture to all these accounts. It is preferably a picture of you, but it can be a drawing of you, or some other avatar you might want to use.
- Send a pull-request to the GitHub repository of the course adding a JSON file. The name of the file should be your GitHub username and it should include key-value pairs as in the example. (The
postswill be an empty list.) Check the result of GitHub Actions.
- Join the Slack workspace (I send invitations to everyone to their email address.) and say hi.
- Write a blog post about the course. In your post link to your GitHub and GitLab accounts and to your Pull-Request. If you encountered any issue, write about that and how you solved it. If you use an avatar instead of your own picture, describe how you created the avatar.
- In the blog post tell us a bit about your background.
- What programming language(s) you use?
- Which interesting 3rd-party libraries do you use? You can mention big ones, but it is probably more interesting if you mention more esoteric ones.
- Include links to the home-page of each project and the GitHub/GitLab repository of each project.
- What would you like to accomplish in the course?
- Which open source projects would you like to contribute to.
Update your Pull-request adding the URL to the blog post to the
postsfield in the json file.
There are 3 GitHub repositories with lists of GitHub organization published by higher education institutions, governments, and corporations. Find at least 5 more organizations that share some of their code using an open source license in GitHub or GitLab. An organization can be a corporation, a university, a college, a research institute, or a government. (e.g. find a list of universities and use the search feature of GitHub to find organizations that belong to the institute).
- The first sessions should be longer. Maybe two parts of 45 minutes and I should cover a number of topics that were now postponed to the 2nd session
Session 2 - Create GitHub Pages using the GitHub UI, Markdown
Show the results so far.
- Our web site
- Some of the blog posts
- Missing project links,
- Missin PRs for the data collection projects
Remind everyone to check the results of the CI and if they don't understand the meaning ask it on the Slack channel.
- If the CI fails, fix it in the same branch. That will update the PR. Do NOT start a new branch.
Why is contributing to Open Source important?
- You received a gift, you give a gift.
- Your code will be checked by others as well.
- You get credit in the open source world and you can also show to your (future) employers.
- These days we like it or not our "brand" has value.
- You get a better product next time you use it (at the same organization or elsewhere).
- You will be able to easily upgrade to the new version of this project.
Why blogging about your process is important?
- improves skills
- helps clarify thoughts
- explaining things to others always helps
- you will be able to look back
- Build your personal "brand".
Show blogging platform
- Show the repo of Flask:
- commits, committers, sha
- show blame of the README file
- Comment on a commit
- Open issue: on our project, on flask there are templates in th
- Show the repo of Flask:
- just view source in a browser
- Bullet points
We mentioned static website generators
Show how to create GitHub pages for the user
username.github.iofirst using only the GitHub web site.
- Show URL that does http://cm-demo.github.io/
- Create repository with README file https://github.com/cm-demo/cm-demo.github.io
- Create docs/index.md with Hello World
- Configure Pages to be served from the docs/ folder
- See the URL now shows the new page.
GitHub flavored Markdown that can be used in Markdown files and elsewhere on GitHub.
- Video 2.2
- Create a web site using GitHub pages with Markdown. At first using the interface on the website of GitHub:
- Create your own GitHub repository
- Create the
docsfolder and in the
docsfolder create a file called
- Include links to your GitHub and GitLab repositories.
- Include links to your blog and to the course at https://osdc.code-maven.com/ .
- Include a picture of you (or if you prefer not to have your picture then a picture you took of something nice.) Describe the source of that picture. Bevare of reusing images you found on the Internet. We don't want you to infringe on the copyright of someone else.
- Create your own GitHub repository
- Add your CV in Markdown
- Create a new file called docs/cv.md
- Start writing your CV there.
- In a blog post describe your experience. Include a link to your new GitHub pages site.
- Link from your new web GitHub pages based web-site to your blogging account and the article you wrote.
Mention all the people from the course whom you know and link to their journal.
Submit by adding the link of your new blog post to your personal JSON file in the project and adding a field called "github_page" to your personal JSON file with a value of "true". See the example of szabgab.json if unclear how.
Session 3 - git client
Show the drawing of the GitHub PR process in the cloud we have used in the previous session.
GitLab: just show that it exists
Show how to use Git on the command line to update the GitHub pages git slides use a new Windows machine for this.
- Git - use the following commands:
git config --global --add user.name "Foo Bar"
git config --global --add user.email firstname.lastname@example.org
These commands created the
Generate private-key public-key pair. Put the public-key in your GitHub user.
sh-keygen Add public key to GitHub
``` git clone
git status git diff git add git commit git show SHA git push git remote -v ```
We also saw:
We also saw some dangerous commands:
git reset HEAD~1 # remove the most recent commit
git push --force # forcibly overwrite what is in the remote
- Clone the repository of your github pages.
- Update the files. (add new files).
- Commit the changes.
- Push the changes out
A couple of suggestions for the blog posts
* Use a title that can sound interesting to others as well eg. How to contribute to an open source project or How to Send a pull request on GitHub.
osdc tag and other relevant tags.
* Add the
series: field to the
Jekyll front matter (the header of each post on DEV.to)
* Use Markdown in the post.
* Include links to the relevant sites and pages such as the web site of the Open Source Development Course and the web site of our course: Open Source Development Course in Hebrew.
Session 4 - Upload your own project - testing
Find the GitHub repositories of packages.
import requestsvia PyPI.
use WWW::Mechanize;via MetaCPAN.
GSONvia Maven Central
Process to add an existing repository to your own GitHub account.
- Create Git repository locally.
- Add files to the local git repository
- Create an empty Git repository on GitHub
- Connect the two,
pushout the local repo.
- Add tests to the project.
Push out the changes.
Generate test coverage report.
Notes from the meeting
- SO with drawing about Git staging
- The slides
- Source of the slides
- Demo repo
- Video 4-2
Update the JSON file and inlcude a list of GitHub and GitLab repositories of projects that you use.
- If you write Python and you have
import requeststhen include the link
- If you write Perl and you have
use WWW::Mechanize;in your code then include
- If you write NodeJS and you have
require('express')in your code then include
https://github.com/vuejs/as it is an organization with several repositories.
- If you write Java and you have in your code then include
- If you write Python and you have
If you don't have any projects on GitHub (GitLab) yet. Upload one.
- If you had a project in an earlier course with me, you can use that.
- If you have a project in your lab that can be published with and open source license (which is not on GitHub yet) then use that.
- If you don't have any personal project yet. Start one now. A few ideas are listed at the bottom of this page.
Include a README file explaining how to use the project.
Once you pushed out the first version to GitHub share the link in our Slack channel.
Monitor the Slack channel and when you see others post their project try to run the code.
- If something is unlcear in the instructions feel free to ask questions in our Slack channel (e.g. in the thread of the annuncement of each project).
- Open an issue on the project you are trying to use to report problems and/or ask for features.
Write a blog post about the work you have done with plenty of links. Add it to your JSON file.
Session 5 - clone, fork, push, pr
- paste link
- edit message
- Add link with text
- Use threads
Go over the few project people have submitted and comment. Can you get the answer to the follow questions? If not, open an issue asking about it.
- What does this project do?
- How do I install dependencies?
- How do I run the tests?
- How do I use this project?
Add requirements to the project of Shuly.
- git checkout -b branch/name
- git push
- git remote add
We used the following to map the name "origin" for the purpuses of
push to the forked repository.
git remote set-url origin --push email@example.com:szabgab/LIMS_results_validation.git
git fetchfollowed by either
~/.gitconfigfile there can be an entry that will set this
This will make
pull = fetch + merge
rebase = false
This will make
pull = fetch + rebase
rebase = true
rebase only impacts the history of the repository and not the code.
Session 6 - testing R, Docker, gitignore, GitHub Actions
Firs we saw an example writing a test in R
Then we had an introduction to Docker
Running a plain Ubuntu-based image:
docker run --rm -it ubuntu:22.04 bash
apt-get install python3
Running an image with Python 3.11 in it:
docker run --rm -it python:3.11
Then we looked at the Dockerfile for mydocker of Gabor
- Docker Images based on Ubuntu
- Docker Images for Python
- We found it has some file in the
__pycache__folder added. Reported it in this issue.
- Then added it to .gitignore PR
git clone firstname.lastname@example.org:zguillez/python-toolz.git
docker run -v/home/gabor/os/python-toolz:/opt --rm -it python:3.11 bash
git remote add fork email@example.com:szabgab/python-toolz.git
We also wrote a test that executed the already existing example and configured GitHub Actions to run it on 3 versions of Python on macOS and on Linux. On Windows the tests failed due to newline issues. We sent a Pull Request with that.
We used the GitHub Actions skeleton for Python.
There are more GitHub Actions skeletons.
Similar to the PyDigger there are also:
- Ruby Digger
- CPAN Digger for Perl
CI Challenge of December 2022 in which every I sent a PR adding GitHub Actions to an open source project written in Python, Perl, or Ruby.
- Video 6-2
- Pick a few projects in your favorite programming language and try to run its tests locally in a Docker container.
- If you encounter any problems, report them. Feel free to first report them in our Slack and report them after we discussed them.
- Files added that should be ignored with .gitignore
- Generated files not listed in .gitignore
- No instructions on how to run tests.
- No tests.
- Tests failing on your local system (inside a Docker is safer).
- If you can set up GitHub Actions.
Alex invited you to look at his project and start contributing there. I'd say look at our web site. You have links to the GitHub and GitLab accounts to each one of the participants of the course. Find something that interests you and try to contribute.
We looked at the source code of DEV.to that is called Forem and is located here in GitHub After some failed searches among the issues we found the discussion Shuly opened, the discussion about auto-save. We also searched the issues and found this issue about the topic. It is unfortuate that the project does not provide enough response to these discussions and issues.
We then discussed the idea of creating a fork and accepting some of the PRs there for our private use. Here are the instruction to do that:
``` git clone URL # clone the project locally git checkout -b mymain # Create a branch that we will maintain. It is better not use the original main or master branches for this. (we use the branch-name szabgab for this in the video)
git fetch origin pull/1299/head # Use the PR number to fetch the code from the pull-request git checkout -b pr-1299 FETCH_HEAD # Create a branch (using the PR number can be a good idea) based on what we have just fetched. git checkout mymain # Go to mymain git merge pr-1299 # Merge the PR into "mymain" ```
There is actually a shorter version:
git fetch origin pull/PR/head:NAME_OF_LOCAL_BRANCH
git fetch origin pull/1351/head:p-1351
First I showed this using a simple PR and it worked.
Then I tried with the one we actually wanted and it had a nasty merge-conflict.
Resolving that would require deep knowledge of the project so we have not tried to do that.
We discuessed that it is probably the responsibilit of the PR-sender to make sure the PR can be merged without a conflict, but then it is very diffictul to do if the PR is not accepted quickly. We saw that there was a merge inside the PR. I mentioned that I would have used rebase there.
- We saw a simple merge conflict resolution.
``` mkdir demo cd demo
git init git add . git commit -m init
git checkout -b first
git add . git commit -m first
git checkout main
git add . git commit -m main
git merge first # this will open the default editor so you can type in the commit message for the merge. ```
Do the above, but this time change the middle line in both branches.
git merge will complain and create a file with markers showing which version came from which branch. You have to edit the file with your favorite editor to resolve the merge conflicts.
There are various GUI-based editors that will make the experience nicer.
However the better solution is to reduce the chances of conflict or even to avoid them. This can be done by working on the same
main branch and pulling and pushing code frequently.
Working in a separate branch is less ideal, but when preparing a PR that's the way to work. In that case try to make the PR as small as possible and try to get it merged as soon as possible. Follow the project and keep rebasing your branch to be up-to-date with the main development branch of the project.
- We mentioned the Dockerfile I use to run projects locally. It can be found here
R-base Docker images for R
testthat for testing R code
How to find projects on GitHub:
- Either sort them by popularity (stars) ot by recent activity: (Recently update)
- Looking at popular project can help understanding how such projects are managed
- Looking at recently update projects with few stars will probably provide more easy opportunities to contribute tests, documentation, CI, etc.
- Projects in R
- Projects in PHP
One thing I'd expect from each project either in its README file or in the CONTRIBUTION file is an explanation
- how to set up the local development environment
- how to install the dependencies of the project (both dependencies in the programming language of the project and potential other dependencies)
- how to run the tests of the project locally
- Ideally there might be a Dockerfile to make it easy to run the whole thing locally.
A sugestion by Alex was: Take a project in some language an convert to another language.
I showed the Kantoniko web site - a multilingual Ladino dictionary.
- Video 7-2
- Set up locally and contribute to Kantoniko and/or PyDigger
- Pick one (or more) open source projects in your favorite language and run the tests locally.
- If you can open an issue, send a PR to these projects
Session 8 planning
- Show git stash
git stash list
git stash show -p (defaults to 0)
git stash show -p 2
git stash pop (defaults to 0)
git stash drop (defaults to 0)
git stash drop 1
We also saw:
- Image: https://hub.docker.com/r/mattrayner/lamp
- Source code: https://github.com/mattrayner/docker-lamp
Pick up a project (Python, R, or PHP), set up the local environemt, run the tests locally.
- Python requests: https://dev.to/szabgab/development-environment-for-the-python-requests-package-eae
- Jinja has a separate file of CONTRIBUTING.rst https://github.com/pallets/jinja
- requests has some information in the README file
- Video 8-2
Session 9 R
Session 10 Overview of the OSDC web site
We also saw the way the web site of each individual course is generated.
Session 11 planning
dev.to mention the setting to use the Markdown editor https://github.com/forem/forem/issues/19214
Pick up a project (Python, R, or PHP), set up the local environment, run the tests locally.
GitHub Actions for each one of them.