Topics

mono-repo & git subtree

espy
 

I've done some poking around at Fede's mono-repo, and also taken a look at some of the available documentation [1] and blog posts [2] about git subtree. Also for reference, I've included a link to Fede's script [3] which uses git-commit to construct the mono repo.

I will have to admit it's a really clean approach, and I think it can be made to work, however I have some questions and comments that I think need to be addressed before we pull the trigger.

1) I *think* the intention here is to use git subtree as a one-time import from the other go repos, and then archive them all, correct? I don't think the intention was that the source repos are maintained and we push/pull things back and forth between the mono repo and the subtree repos.

2) One interesting thing I discovered, is that git subtree preserves all the commit metadata from the sub-tree repos. It does this by using some low-level git command (e.g. git commit-tree) which are normally not used by regular users.

3) git subtree offers the ability to squash all the commit messages from the source repos, but it looks like we're making a conscious decision to preserve the commit history.

4) I'm not a big fan of the default commit messages that are generated:

commit 3f338c32b3a29c1d6f474246ec11fff09afb42bb
Merge: c477ab7 c5d3809
Author: Federico Claramonte <fede.claramonte@...>
Date: Fri Jan 26 09:36:56 2018 +0100

Add 'core/command/' from commit 'c5d38095ff88b214ad22de001d0fa8e400efd61b'

git-subtree-dir: core/command
git-subtree-mainline: c477ab724c1d7397c188a332f761b750b7f25946
git-subtree-split: c5d38095ff88b214ad22de001d0fa8e400efd61b

First, the message summary is >50 chars. I think a better summary would be something like:

Import from github.com/edgexfoundry/core-command-go.git

This is still longer than 50 chars, but exceptions are OK, especially for something like this. The other alternative is to include the
source repo in the commit message body. Without this, it's hard to tell where the code was actually imported from.

I also have some issues with the commit message body. First, I'd like to see us use shortened SHA1's (eg. "c5d3809").

"git-subtree-mainline:" always seems to be the previous merge commit, and doesn't really add much value. Also the short versions of both "git-subtree-mainline" and "git-subtree-split" are shown by the "Merge" line that's output by git log.

Finally, it looks like git subtree offers control over the commit message, so these aren't issues with the overall approach, and instead just changes to how the tool is used, in a way that makes the history easier to grok.

Thoughts?

Regards,
/tony

p.s. nice work Fede!


[1] https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt

[2] https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree

[3] https://gist.github.com/feclare/8dba191e8cf77864fe5eed38b380f13a

James.White2@...
 

Tony,

thanks as always for your considered and detailed review.

Yes - agree I think Fede has done some good work (thanks Fede)!

1) you have the intention correct. Once the source repos are pulled into the main repo by the script, and we are satisfied with its structure and content, we archive the old source repo. There may be a small period where developers have to make dual commits to old and new mono repo, but we'll work to keep this minimal.
2) It seemed this was just an observation, but wasn't sure if you are recommending something I missed.
3) Yes again - are you suggesting we don't need the commit tree? I like the idea of preserving it.
4) I am ok with your suggested changes to commit messages so long as there are not strong community push back and (more importantly) Fede is ok with making those changes.

jim
________________________________________
From: edgex-golang-bounces@... <edgex-golang-bounces@...> on behalf of espy <espy@...>
Sent: Tuesday, February 6, 2018 7:41 PM
To: edgex-golang@...
Subject: [Edgex-golang] mono-repo & git subtree

I've done some poking around at Fede's mono-repo, and also taken a look
at some of the available documentation [1] and blog posts [2] about git
subtree. Also for reference, I've included a link to Fede's script [3]
which uses git-commit to construct the mono repo.

I will have to admit it's a really clean approach, and I think it can be
made to work, however I have some questions and comments that I think
need to be addressed before we pull the trigger.

1) I *think* the intention here is to use git subtree as a one-time
import from the other go repos, and then archive them all, correct? I
don't think the intention was that the source repos are maintained and
we push/pull things back and forth between the mono repo and the subtree
repos.

2) One interesting thing I discovered, is that git subtree preserves all
the commit metadata from the sub-tree repos. It does this by using some
low-level git command (e.g. git commit-tree) which are normally not used
by regular users.

3) git subtree offers the ability to squash all the commit messages from
the source repos, but it looks like we're making a conscious decision to
preserve the commit history.

4) I'm not a big fan of the default commit messages that are generated:

commit 3f338c32b3a29c1d6f474246ec11fff09afb42bb
Merge: c477ab7 c5d3809
Author: Federico Claramonte <fede.claramonte@...>
Date: Fri Jan 26 09:36:56 2018 +0100

Add 'core/command/' from commit
'c5d38095ff88b214ad22de001d0fa8e400efd61b'

git-subtree-dir: core/command
git-subtree-mainline: c477ab724c1d7397c188a332f761b750b7f25946
git-subtree-split: c5d38095ff88b214ad22de001d0fa8e400efd61b

First, the message summary is >50 chars. I think a better summary would
be something like:

Import from github.com/edgexfoundry/core-command-go.git

This is still longer than 50 chars, but exceptions are OK, especially
for something like this. The other alternative is to include the
source repo in the commit message body. Without this, it's hard to tell
where the code was actually imported from.

I also have some issues with the commit message body. First, I'd like to
see us use shortened SHA1's (eg. "c5d3809").

"git-subtree-mainline:" always seems to be the previous merge commit,
and doesn't really add much value. Also the short versions of both
"git-subtree-mainline" and "git-subtree-split" are shown by the "Merge"
line that's output by git log.

Finally, it looks like git subtree offers control over the commit
message, so these aren't issues with the overall approach, and instead
just changes to how the tool is used, in a way that makes the history
easier to grok.

Thoughts?

Regards,
/tony

p.s. nice work Fede!


[1] https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt

[2]
https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree

[3] https://gist.github.com/feclare/8dba191e8cf77864fe5eed38b380f13a
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

espy
 

On 02/06/2018 10:56 PM, James.White2@... wrote:
Tony,
thanks as always for your considered and detailed review.
Yes - agree I think Fede has done some good work (thanks Fede)!
1) you have the intention correct. Once the source repos are pulled into the main repo by the script, and we are satisfied with its structure and content, we archive the old source repo. There may be a small period where developers have to make dual commits to old and new mono repo, but we'll work to keep this minimal.
Great!

2) It seemed this was just an observation, but wasn't sure if you are recommending something I missed.
Correct, just an observation. As I mentioned yesterday, the first device-bacnet PR I reviewed involved copying code from one edgex repo to another, and preserving provenance of the copied files was something I wanted to ensure. I poked under the covers of git subtree because I wanted to understand how it actually work, especially wrt preserving the original commit hashes from the source tree.

3) Yes again - are you suggesting we don't need the commit tree? I like the idea of preserving it.
Not suggesting a change, just wanted to ask if there'd been any discussion about squashing vs. preserving. I'm +1 keeping it as is.

4) I am ok with your suggested changes to commit messages so long as there are not strong community push back and (more importantly) Fede is ok with making those changes.
The only change I feel strongly about addition of an explicit reference to the *actual* subtree source git repository that is being merged. All the other changes were suggestions that just align with our commit style and clarify what the merge actually represents, so are less important (but helpful IMHOP).

/t


jim
________________________________________
From: edgex-golang-bounces@... <edgex-golang-bounces@...> on behalf of espy <espy@...>
Sent: Tuesday, February 6, 2018 7:41 PM
To: edgex-golang@...
Subject: [Edgex-golang] mono-repo & git subtree
I've done some poking around at Fede's mono-repo, and also taken a look
at some of the available documentation [1] and blog posts [2] about git
subtree. Also for reference, I've included a link to Fede's script [3]
which uses git-commit to construct the mono repo.
I will have to admit it's a really clean approach, and I think it can be
made to work, however I have some questions and comments that I think
need to be addressed before we pull the trigger.
1) I *think* the intention here is to use git subtree as a one-time
import from the other go repos, and then archive them all, correct? I
don't think the intention was that the source repos are maintained and
we push/pull things back and forth between the mono repo and the subtree
repos.
2) One interesting thing I discovered, is that git subtree preserves all
the commit metadata from the sub-tree repos. It does this by using some
low-level git command (e.g. git commit-tree) which are normally not used
by regular users.
3) git subtree offers the ability to squash all the commit messages from
the source repos, but it looks like we're making a conscious decision to
preserve the commit history.
4) I'm not a big fan of the default commit messages that are generated:
commit 3f338c32b3a29c1d6f474246ec11fff09afb42bb
Merge: c477ab7 c5d3809
Author: Federico Claramonte <fede.claramonte@...>
Date: Fri Jan 26 09:36:56 2018 +0100
Add 'core/command/' from commit
'c5d38095ff88b214ad22de001d0fa8e400efd61b'
git-subtree-dir: core/command
git-subtree-mainline: c477ab724c1d7397c188a332f761b750b7f25946
git-subtree-split: c5d38095ff88b214ad22de001d0fa8e400efd61b
First, the message summary is >50 chars. I think a better summary would
be something like:
Import from github.com/edgexfoundry/core-command-go.git
This is still longer than 50 chars, but exceptions are OK, especially
for something like this. The other alternative is to include the
source repo in the commit message body. Without this, it's hard to tell
where the code was actually imported from.
I also have some issues with the commit message body. First, I'd like to
see us use shortened SHA1's (eg. "c5d3809").
"git-subtree-mainline:" always seems to be the previous merge commit,
and doesn't really add much value. Also the short versions of both
"git-subtree-mainline" and "git-subtree-split" are shown by the "Merge"
line that's output by git log.
Finally, it looks like git subtree offers control over the commit
message, so these aren't issues with the overall approach, and instead
just changes to how the tool is used, in a way that makes the history
easier to grok.
Thoughts?
Regards,
/tony
p.s. nice work Fede!
[1] https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt
[2]
https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree
[3] https://gist.github.com/feclare/8dba191e8cf77864fe5eed38b380f13a
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Fede Claramonte
 

I have updated the script with some of your suggestions:
- Changing the commit message of the git-subtree commands, but only allows to change the first line. Changing the remaining commit lines should need to modify the git-subtree.sh
- Automated all the commits in the script, so it can be run again to update the repo with latest PR in the go repos. The script create the repo again, so any manual commit will be lost.
- Changed version files to export/distro and export/client instead of cmd/client cmd/distro.

The current output of the script could be find in my repo:
https://github.com/feclare/edgex-go/commits/monorepo_clean

Fede

On 07/02/18 14:07, espy wrote:
On 02/06/2018 10:56 PM, James.White2@... wrote:
Tony,

thanks as always for your considered and detailed review.

Yes - agree I think Fede has done some good work (thanks Fede)!

1) you have the intention correct.  Once the source repos are pulled into the main repo by the script, and we are satisfied with its structure and content, we archive the old source repo. There may be a small period where developers have to make dual commits to old and new mono repo, but we'll work to keep this minimal.
Great!

2) It seemed this was just an observation, but wasn't sure if you are recommending something I missed.
Correct, just an observation.  As I mentioned yesterday, the first device-bacnet PR I reviewed involved copying code from one edgex repo to another, and preserving provenance of the copied files was something I wanted to ensure.  I poked under the covers of git subtree because I wanted to understand how it actually work, especially wrt preserving the original commit hashes from the source tree.

3) Yes again - are you suggesting we don't need the commit tree?  I like the idea of preserving it.
Not suggesting a change, just wanted to ask if there'd been any discussion about squashing vs. preserving.  I'm +1 keeping it as is.

4) I am ok with your suggested changes to commit messages so long as there are not strong community push back and (more importantly) Fede is ok with making those changes.
The only change I feel strongly about addition of an explicit reference to the *actual* subtree source git repository that is being merged.  All the other changes were suggestions that just align with our commit style and clarify what the merge actually represents, so are less important (but helpful IMHOP).

/t



jim
________________________________________
From: edgex-golang-bounces@... <edgex-golang-bounces@...> on behalf of espy <espy@...>
Sent: Tuesday, February 6, 2018 7:41 PM
To: edgex-golang@...
Subject: [Edgex-golang] mono-repo & git subtree

I've done some poking around at Fede's mono-repo, and also taken a look
at some of the available documentation [1] and blog posts [2] about git
subtree.  Also for reference, I've included a link to Fede's script [3]
which uses git-commit to construct the mono repo.

I will have to admit it's a really clean approach, and I think it can be
made to work, however I have some questions and comments that I think
need to be addressed before we pull the trigger.

1) I *think* the intention here is to use git subtree as a one-time
import from the other go repos, and then archive them all, correct?  I
don't think the intention was that the source repos are maintained and
we push/pull things back and forth between the mono repo and the subtree
repos.

2) One interesting thing I discovered, is that git subtree preserves all
the commit metadata from the sub-tree repos.  It does this by using some
low-level git command (e.g. git commit-tree) which are normally not used
by regular users.

3) git subtree offers the ability to squash all the commit messages from
the source repos, but it looks like we're making a conscious decision to
preserve the commit history.

4) I'm not a big fan of the default commit messages that are generated:

commit 3f338c32b3a29c1d6f474246ec11fff09afb42bb
Merge: c477ab7 c5d3809
Author: Federico Claramonte <fede.claramonte@...>
Date:   Fri Jan 26 09:36:56 2018 +0100

      Add 'core/command/' from commit
'c5d38095ff88b214ad22de001d0fa8e400efd61b'

      git-subtree-dir: core/command
      git-subtree-mainline: c477ab724c1d7397c188a332f761b750b7f25946
      git-subtree-split: c5d38095ff88b214ad22de001d0fa8e400efd61b

First, the message summary is >50 chars.  I think a better summary would
be something like:

Import from github.com/edgexfoundry/core-command-go.git

This is still longer than 50 chars, but exceptions are OK, especially
for something like this.  The other alternative is to include the
source repo in the commit message body. Without this, it's hard to tell
where the code was actually imported from.

I also have some issues with the commit message body. First, I'd like to
see us use shortened SHA1's (eg. "c5d3809").

"git-subtree-mainline:" always seems to be the previous merge commit,
and doesn't really add much value.  Also the short versions of both
"git-subtree-mainline" and "git-subtree-split" are shown by the "Merge"
line that's output by git log.

Finally, it looks like git subtree offers control over the commit
message, so these aren't issues with the overall approach, and instead
just changes to how the tool is used, in a way that makes the history
easier to grok.

Thoughts?

Regards,
/tony

p.s. nice work Fede!


[1] https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt

[2]
https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree

[3] https://gist.github.com/feclare/8dba191e8cf77864fe5eed38b380f13a
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

James.White2@...
 

Thanks Fede!
________________________________________
From: edgex-golang-bounces@... <edgex-golang-bounces@...> on behalf of Fede Claramonte <fclaramonte@...>
Sent: Wednesday, February 7, 2018 7:42 AM
To: edgex-golang@...
Subject: Re: [Edgex-golang] mono-repo & git subtree

I have updated the script with some of your suggestions:
- Changing the commit message of the git-subtree commands, but only
allows to change the first line. Changing the remaining commit lines
should need to modify the git-subtree.sh
- Automated all the commits in the script, so it can be run again to
update the repo with latest PR in the go repos. The script create the
repo again, so any manual commit will be lost.
- Changed version files to export/distro and export/client instead of
cmd/client cmd/distro.

The current output of the script could be find in my repo:
https://github.com/feclare/edgex-go/commits/monorepo_clean

Fede

On 07/02/18 14:07, espy wrote:
On 02/06/2018 10:56 PM, James.White2@... wrote:
Tony,

thanks as always for your considered and detailed review.

Yes - agree I think Fede has done some good work (thanks Fede)!

1) you have the intention correct. Once the source repos are pulled
into the main repo by the script, and we are satisfied with its
structure and content, we archive the old source repo. There may be a
small period where developers have to make dual commits to old and
new mono repo, but we'll work to keep this minimal.
Great!

2) It seemed this was just an observation, but wasn't sure if you are
recommending something I missed.
Correct, just an observation. As I mentioned yesterday, the first
device-bacnet PR I reviewed involved copying code from one edgex repo
to another, and preserving provenance of the copied files was
something I wanted to ensure. I poked under the covers of git subtree
because I wanted to understand how it actually work, especially wrt
preserving the original commit hashes from the source tree.

3) Yes again - are you suggesting we don't need the commit tree? I
like the idea of preserving it.
Not suggesting a change, just wanted to ask if there'd been any
discussion about squashing vs. preserving. I'm +1 keeping it as is.

4) I am ok with your suggested changes to commit messages so long as
there are not strong community push back and (more importantly) Fede
is ok with making those changes.
The only change I feel strongly about addition of an explicit
reference to the *actual* subtree source git repository that is being
merged. All the other changes were suggestions that just align with
our commit style and clarify what the merge actually represents, so
are less important (but helpful IMHOP).

/t



jim
________________________________________
From: edgex-golang-bounces@...
<edgex-golang-bounces@...> on behalf of espy
<espy@...>
Sent: Tuesday, February 6, 2018 7:41 PM
To: edgex-golang@...
Subject: [Edgex-golang] mono-repo & git subtree

I've done some poking around at Fede's mono-repo, and also taken a look
at some of the available documentation [1] and blog posts [2] about git
subtree. Also for reference, I've included a link to Fede's script [3]
which uses git-commit to construct the mono repo.

I will have to admit it's a really clean approach, and I think it can be
made to work, however I have some questions and comments that I think
need to be addressed before we pull the trigger.

1) I *think* the intention here is to use git subtree as a one-time
import from the other go repos, and then archive them all, correct? I
don't think the intention was that the source repos are maintained and
we push/pull things back and forth between the mono repo and the subtree
repos.

2) One interesting thing I discovered, is that git subtree preserves all
the commit metadata from the sub-tree repos. It does this by using some
low-level git command (e.g. git commit-tree) which are normally not used
by regular users.

3) git subtree offers the ability to squash all the commit messages from
the source repos, but it looks like we're making a conscious decision to
preserve the commit history.

4) I'm not a big fan of the default commit messages that are generated:

commit 3f338c32b3a29c1d6f474246ec11fff09afb42bb
Merge: c477ab7 c5d3809
Author: Federico Claramonte <fede.claramonte@...>
Date: Fri Jan 26 09:36:56 2018 +0100

Add 'core/command/' from commit
'c5d38095ff88b214ad22de001d0fa8e400efd61b'

git-subtree-dir: core/command
git-subtree-mainline: c477ab724c1d7397c188a332f761b750b7f25946
git-subtree-split: c5d38095ff88b214ad22de001d0fa8e400efd61b

First, the message summary is >50 chars. I think a better summary would
be something like:

Import from github.com/edgexfoundry/core-command-go.git

This is still longer than 50 chars, but exceptions are OK, especially
for something like this. The other alternative is to include the
source repo in the commit message body. Without this, it's hard to tell
where the code was actually imported from.

I also have some issues with the commit message body. First, I'd like to
see us use shortened SHA1's (eg. "c5d3809").

"git-subtree-mainline:" always seems to be the previous merge commit,
and doesn't really add much value. Also the short versions of both
"git-subtree-mainline" and "git-subtree-split" are shown by the "Merge"
line that's output by git log.

Finally, it looks like git subtree offers control over the commit
message, so these aren't issues with the overall approach, and instead
just changes to how the tool is used, in a way that makes the history
easier to grok.

Thoughts?

Regards,
/tony

p.s. nice work Fede!


[1]
https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt

[2]
https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree


[3] https://gist.github.com/feclare/8dba191e8cf77864fe5eed38b380f13a
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang