The Fugue Counterpoint by Hans Fugal

10Nov/0840

git-push is worse than worthless

Ugh

Let's say you are a web developer, and you do development on your laptop, then when things are nice and shiny you want to push those changes to the webserver. Seems natural enough, right?

git clone server:/var/www/foo
# ...
git pull

# edit stuff
git commit -a -m 'i edited stuff'

Now, let's say you're a (possibly former) darcs/bzr/mercurial user and this time you're using git. Git has git-push. You read the man page like a good little code monkey, and it seems like it does the same or similar thing to darcs push or hg push. It seems like if you want to push your changes to the server, you'd do this:

git push

Am I off in left field or does this not seem 100% rational? But wo be unto the code monkey that utters this unfortunate incantation. Observe:

$ mkdir foo
$ cd foo
$ git init
Initialized empty Git repository in /private/tmp/foo/.git/
$ echo hello > foo.txt
$ git add foo.txt
$ git commit -m 'hello'
Created initial commit bee50da: hello
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 foo.txt
$ cd ..
$ git clone foo bar
Initialized empty Git repository in /private/tmp/bar/.git/
$ cd bar
$ echo goodbye >> foo.txt
$ git commit -a -m goodbye
Created commit 99c13c1: goodbye
 1 files changed, 1 insertions(+), 0 deletions(-)
$ git push ../foo
Counting objects: 5, done.
Writing objects: 100% (3/3), 248 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To ../foo
   bee50da..99c13c1  master -> master
$ cd ../foo
$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#   modified:   foo.txt
#
$ git diff
$ git diff --cache
error: invalid option: --cache
$ git diff --cached
diff --git a/foo.txt b/foo.txt
index a32119c..ce01362 100644
--- a/foo.txt
+++ b/foo.txt
@@ -1,2 +1 @@
 hello
-goodbye
$ git log
commit 99c13c1e60888ae2c0e221898411e1cd52ad3815
Author: Hans Fugal <hans@fugal.net>
Date:   Mon Nov 10 17:11:57 2008 -0700

    goodbye

commit bee50da72798edc47ddc36dbc4f559f141b1e28b
Author: Hans Fugal <hans@fugal.net>
Date:   Mon Nov 10 17:11:34 2008 -0700

    hello

I promise I didn't fake that. Yes, you saw that correctly—git wants to undo the changes you just committed. If you happen to have a clean working directory, all you need to do to return to sanity is git reset HEAD. If not, heaven help you.

This is totally unacceptable. It's unforgivable on so many levels. At the very least, the manpage should warn you to not push to repositories with working copies. Git should warn you before you push and screw up your repo that it has a working copy checked out. Ideally, git would behave like darcs and update the working copy. Suboptimally, it would behave like mercurial and make it a new revision that you have to manually checkout. But this is simply ridiculous.

So what is the solution? They tell you to use pull. Hello! Is anyone home? My laptop is roving. It's often behind a NATing firewall. I'm supposed to find my public IP address and figure out how to subvert the evil firewalls of the world every time I want to push my changes to the server?

A workaround, and probably the best real-world workflow, is to have a second bare repository or a second branch on the server, push into that, then ssh into the server and pull the changes. I think this page describes how to do that with a second branch, though I'm short on time to actually try it out at the moment.

More of this sickening story in this thread, where you will learn that at least one other person out there has his head screwed on properly, that the developers are more interested in how hooks work (and fail to allow you to do this even if you grok them), and that they've discussed the problem before and decided the correct response is to RTFM (M for minds this time, since the manual was completely unhelpful).

Update: Some of you have been quick to defend git and the design choice of how push behaves. I want to clarify that I don't care so much that push updates the repository but not the working directory. Mercurial works this way too. Not the way I'd do it but it's a valid approach. The problem here is that git push seems like a natural thing to do but screws up your working directory on the remote side. Mercurial doesn't change the working directory, but neither does it silently rebase it and set you up to undo your changes if you're not careful. The problem here is a lack of safety and a lack of warning. They know it's a problem, they've fielded enough "morons". A few words of warning in the man page is all it would take to make me happy.

And now, I have had time to work out a more specific workaround. Here's what I did, and it seems to work well:

# Server setup (set up incoming branch)
server$ git branch incoming
server$ git branch
  incoming
* master
  origin

# Laptop setup (local master to remote incoming)
laptop$ git config remote.origin.push master:incoming

# Everyday usage
laptop$ git push
Counting objects: 5, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 279 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
To server:/tmp/foo
   b108a07..a9d3282  master -> incoming

server$ git status
# On branch master
nothing to commit (working directory clean)
server$ git pull . incoming
From .
 * branch            incoming   -> FETCH_HEAD
Updating b108a07..a9d3282
Fast forward
 foo.txt |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

You could use hooks to automatically do that git pull . incoming if you liked, making it more like darcs than mercurial.

Updated update: On further thought, the cleanest solution is probably to have a separate master (bare) repository, e.g.

$ mkdir master
$ cd master
$ git init
Initialized empty Git repository in /private/tmp/foo/master/.git/
$ git commit --allow-empty -m initial
Created initial commit 999755e: initial
$ cd ..
$ git clone master live
Initialized empty Git repository in /private/tmp/foo/live/.git/
$ cd live
$ git branch
* master
$ cd ..
$ git clone master laptop
Initialized empty Git repository in /private/tmp/foo/laptop/.git/
$ cd laptop
$ echo hello > foo.txt
$ git add foo.txt
$ git commit -m hello
Created commit 2297bcf: hello
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 foo.txt
$ git push
Counting objects: 4, done.
Writing objects: 100% (3/3), 239 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /tmp/foo/master/.git
   999755e..2297bcf  master -> master
$ cd ../live
$ ls
$ git pull
remote: Counting objects: 4, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /tmp/foo/master/
   999755e..2297bcf  master     -> origin/master
Updating 999755e..2297bcf
Fast forward
 foo.txt |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 foo.txt
$ echo goodbye >> foo.txt
$ git commit -a -m goodbye
Created commit 04f6702: goodbye
 1 files changed, 1 insertions(+), 0 deletions(-)
$ git push
Counting objects: 5, done.
Writing objects: 100% (3/3), 248 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /tmp/foo/master/.git
   2297bcf..04f6702  master -> master
$ cd ../laptop
$ git pull
remote: Counting objects: 5, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /tmp/foo/master/
   2297bcf..04f6702  master     -> origin/master
Updating 2297bcf..04f6702
Fast forward
 foo.txt |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
Comments (40) Trackbacks (4)
  1. I don’t think I follow 100%. Can’t you just rebase, resolve your conflicts if neccessary, and then push?

    I’m not entirely sure why you would push from one local repo to another local repo.

  2. Hi Mike,

    The example is local just to demonstrate, because it was easier to record (using script) the example and behaves the same way.

    In the real world, I ended up having to muck with the index to get git to forget about the undo changes, then do a revert. I think this is because on the server an older version of git (1.4 vs 1.6 here) is behaving slightly different from what you see in my example. A rebase might also work as a way to dig yourself out of the hole if you accidentally do this.

    The big deal isn’t that push behaves this way. It’s a design choice, and one I can live with. But it’s a counterintuitive choice for people coming from other DRCSes, and so it warrants at least a warning in the man page and preferably a refusal to screw up your working copy (without a –force).

  3. Don’t sad codemonkey!
    This feature (it is a feature) puzzled me for awhile too. As a web dev it’s very useful to push your local (laptop) changes to a live copy on a server, but the remote origin working copy doesn’t update.

    Check out http://git.or.cz/gitwiki/GitFaq#head-b96f48bc9c925074be9f95c0fce69bcece5f6e73 for a solution. Download the post-update hook to $REPO/.git/hooks and chmod +x it. Now the working copy gets updated!

  4. That’s a good link, thanks. I think I would like to make two points in response to your comment. First, yes it is useful as a web developer, but also in many other normal situations. I realize it doesn’t really come up in kernel development (where they send patches by email) but that doesn’t make it an irrelevant way to work for many development situations, not just web development (which is a kind of minority discriminated against by many “serious” programmers, and nothing is more “serious” than kernel development).

    The second thing I disagree with is that it’s a feature. The feature is that they don’t want to update the working directory because they don’t know if they can safely do that. That’s ok. Mercurial does the same thing—you have to update your working directory manually (or set up a hook). Darcs takes a slightly different but just as safe approach: it refuses to do the push if it’s not safe. Either way, no big deal.

    Where git gets it wrong is by yanking the HEAD out from under the working directory and replacing it with a different HEAD, without updating the working directory and without warning the user.

    I have heard a few times that “push is the mirror of fetch”. I encourage you to try git fetch followed by a git status and see if your working directory is now out of whack. It’s not, because you have to manually merge the fetch (which updates your working directory) with git merge FETCH_HEAD. If git push used a PUSH_HEAD, that would be a mirror operation, and I would not consider it broken in any way.

  5. Thanks for this. I was wondering about whats going on for a while. I agree that this is more like a bug than a feature.

  6. Your updated update is the workflow I follow. Thanks for the write-up.

  7. Thank you! That’ exactly what I needed!

  8. I’m trying to understand this.

    I have a client machine A and server machine B, I want to send my client changes to my server.
    My client A can only connect to my server, so I tried git push to send the changes up, it didn’t work, then I read your article and felt more lost………..
    I try to read it again.

  9. Well I read your article on the bus, now I got it – its amazingly stupid because everyone is going to think that git push is going to put my stuff across to the other machine and then some command on the other machine is going to re-integrate those changes back into the files.

    Someone just set a trap for 10 thousand newbies to git.

    I’m a subversion person, so I’m trying to work out whether to move over to git…. let me look at your other article.

  10. I feel your pain. I hate this so-called feature as well. a coworker said my best bet is to create a bare repo in server, and then push my local repo on my laptop to it, and then have a working tree on the server pull from the bare repo. seems like a lot of work for something that seem to be logically trivial… :(

  11. Thanks for this write-up. This was the final (famous last words) setup that I needed to get my laptop setup for development. I’m running ubuntu on virtual-box, and I’m using git and ssh to update my django site on Dreamhost. Thanks, man! Now I can develop anytime anywhere!

  12. Eric: thanks for the heads up warning, and the updates.

    Hans: thanks for the link.

    The whole story finally made sense when reading:
    http://git.or.cz/gitwiki/GitFaq#push-is-reverse-of-fetch
    “Reverse” is not the most adequate term.

    It’s funny how the “updated update” ends up implicitly rebuilding a centralized solution.

  13. Ah, so it seems it wasn’t me doing something wrong. I was bitten by this ‘feature’ yesterday. Thanks for writing it out in detail! Now I have some good starting points to figure out the correct workaround for my special case.

  14. I wonder if there is some hacks to emulate a : git update command ?
    Since update the keyword is not used , what would you suggest it do ?


    http://rzr.online.fr/q/git

  15. Thanks very much for this; I was wondering exactly how I should push changes from my laptop to my desktop. I ended up going with an incoming branch, since it’s pretty simple and it seems to work well.

  16. I spent an hour or so agonizing over what was going on here as well. I like almost everything about Git, but this is -really- unintuitive.

    If I’m reading everything correctly, what is happening is that HEAD is “silently” being changed underneath you, but your altered working tree files aren’t affected. Thus, it looks like Git wants to undo everything that was just pushed. Does that sound right?

    If that’s the case, can you just “update” by doing a git checkout HEAD -f ? Or would that have side effects I’m not seeing?

  17. Dave, you’ve got it right. Yes, you can update with git checkout or git reset. The caveat is if you’ve made changes to your local working directory (or make changes after the push but before doing the reset to the new HEAD), you would lose those changes.

  18. Strangely enough, it is precisely because of this issue that I moved AWAY from Git towards Mercurial. (That, and Windows performance issues.)

    I’ve never been happier.

  19. Hmm i don’t understand this post, because if you push to origin, origin is NOT updated, so live webpage is not affected. You have to reset –hard HEAD if you want see this changes, but you will loose all your changes on live webpage, better solution is to create another branch on live webpage push there any changes and use git merge on live webpage … so you have all changes under control

    i love git, because ist extremly fast compere to svn or any other version system … with project of about 100 000 code lines is realy fast. i had headache with svn on that project, git solve my problem, the speed, the easy way to find bug code, the size of repository (svn was huge after some time), the git grep (excelent function if you want find some string in code).

    Bye

  20. I got bitten by this too just now… Very annoying, ended up having to throw some local changes away to get the repo back to a sane state :(

  21. I agree that the docs for git push are misleading, and Git certainly doesn’t warn you if you’re doing anything “stupid”. I did exactly the same thing when I was learning Git – I pushed to a repo on another machine that had a checked-out tree, and was confused when I did “git status” on the remote.

    Some comments:

    (1) Since git push only transfers objects and moves the remote head (it doesn’t touch the index or the working tree – how could it? since a bare repo has neither) – you should be able to recover by doing a “git reset –soft HEAD^” – which will back up one commmit (the one accidentally pushed) and leave the index and tree alone.

    (2) “git fetch thathost” – the way we bring new objects and heads (from thathost) into a repo (on thishost) that has an index and a working tree – only ever updates heads in thishost’s refs/remotes/thathost/* – not in refs/heads/*. So the “opposite” command – “git push thathost” – assuming thathost also contains a working tree and index – should update thathost’s refs/remotes/thishost/* heads – and not its refs/heads/*. I’ve done this _lots_ and it works great.

    Of course, if you set up a remote using “git remote add” it sets up to push by default into thathost’s refs/heads/* – tacitly assuming the remote to be a bare repo.

    (3) “git init” should add a hook that refuses pushes to refs/heads/*. We only ever push this way into –bare repos.

    And perhaps “git remote add” should ask if the remote is bare or not.

    Does that all make sense?

  22. David, excellent points all. I am now fully convinced that the proper thing for git push to do is as you outlined in (2).

    Thankfully, recent versions of git are much more proactive about warning you about this, and require you to confirm that you really want to unwisely push to a working directory.

  23. I juts got the warning about this and googled around and found this post. I totally agree that this is broken. I already have enough trouble keeping track of all the git repos that might have changes, this just means everyone needs yet another one on their main machine just so they can push back from their laptops! And they have to remember to check that bare repo when they get back into the office, and so on… it’s just not sustainable. For a supposedly distributed version control system, this seems at best a really odd choice. As someone else said, basically with the extra bare repo you’re just reinventing the centralized model.

  24. Git from version 1.7.0 would (by default) refuse to push to currently checked out branch. By default it means that you can configure this behavior.

    Note also that all tutorials, user manuals, etc. tell you to not push into non-bare repository (i.e. one with working directory attached)!

  25. I’ve written up the steps needed to set up a bare repository which makes all of this pretty simple.

    http://craiccomputing.blogspot.com/2010/03/git-push-and-pull-between-repositories.html

    Like much of what we do, it’s easy once you know – but getting there can be a pain.

  26. Thanks for this post. I’m sure many people are bitten by this ‘feature’ and just walk away. I kind of get the logic, but it’s very surprising.
    I never work directly on the server so ‘losing changes’ made there isn’t a relevant concept. On my laptops I have a number of scripts that I run to check for updated-ness of git repos and for ‘publish’. The last stage of publish is
    [code]
    ssh server_name "cd $REMOTE_DIR && git checkout -f"
    [/code]
    I’ve been aware for a while that a bare server repo with working copies on laptop and server is better, but this works for me so “que sera sera”.

    BTW, a handy incantation for updating your local repo status, assuming all your code is in trees under ~/code:

    [CODE]
    $ find ~/code -type d -name .git -exec sh -c ‘D=`echo {} | sed -e “s,/.git,,”`; echo $D; cd $D; git pull; cd – >/dev/null’ \;
    [/CODE]

    Cheers, al.

  27. Hi everyone, this post is beginning to show its age. Thanks for all the links, comments, etc.

    Recent versions of git now refuse to push to a working directory with a nice explanation. I don’t know if my post had any influence on making that a reality, but as users we can claim victory nonetheless!

    The biggest use case for git push is laptop/dev server. The laptop is behind various firewalls at any given time, and the dev server can’t pull from it. I’ve found two highly successful ways of dealing with this. The first is to set up ssh to automatically set up a tunnel back to your laptop, so you can pull over the tunnel. This works quite well in practice, but is slightly noisy (ssh will complain a bit about not being able to set up the tunnel if the tunnel already exists).

    The second approach is to set up repositories on the laptop to push to a prefix, making push symmetric with fetch. e.g. in .git/config

    push = +refs/heads/*:refs/remotes/laptop/*

    If your laptop is sometimes available for pull (e.g. with an ssh tunnel or a non-firewall connection some of the time), then this works rather well. Otherwise it builds up cruft that you can’t get rid of with “git remote prune”, and you have to “rm -r .git/refs/remotes/laptop” and then repush from the laptop, which might make some queasy.

  28. I think the best solution is not clear cut. The behaviour you observe can seem counter intuitive but the question is what are the other options? I can only really think of one that makes sense. To have the remote machine end up on a temporary branch whose HEAD is the commit they were basing their work off. They would then be free to rebase their work off of the master branch. When they were happy, they’d be able to merge their changes into the master branch. This avoids the typical confusion and allows as many pushes as you’d like to take place before you rebase without having to remember anything special. The only concern is the user not noticing they are no longer on the master, but I guess you could get around this with a warning when they next execute a git command “you’re now on branch ‘old_master’ instead of ‘master’ due to a push”. In this way you never see strange things in “git status” and you have as much flexibility as with “git fetch ; git rebase”. In fact in writing this I’ve convinced myself that it’s the correct approach. Can anyone see a reason why this wouldn’t work?

  29. Dan, the fundamental problem with that line of reasoning is that you assume that a push should change the working state of the remote repository in a way that is compatible with *not* changing the working state of the remote repository. This is just silliness. Either go whole hog and push the full state (refusing if the checkout isn’t clean), or don’t mess with the working checkout at all (mirror operation to git fetch) and let the user do the merge or rebase. I think the latter is more git-like. I think the former is fine too. But I think the current behavior is completely broken. (Although, these days it does spit out a big warning with instructions and maybe even refuses to push to a non-bare repository without overriding. So we’ve come a long way.)

  30. Well.. I always want several things to happen with an operation like push.

    1. The remote and local repositories to end up in a meaningful state (ie they have all the commits).
    2. The remote and local checkouts to end up in a sensible state (they have all their local changes, considered to be made against the relevant parent).
    3. The remote and local user be aware that the changes have occurred (I won’t necessarily be the remote user, nor am I necessarily in contact with him).

    The concern I have with mirroring fetch is that when you fetch you know you’ve done it, and that it makes sense to rebase… If push effectively did a fetch on the remote machine the user of the remote repo wouldn’t necessarily know that they should be rebasing or that they’d been pushed to. I suspect we have different workflows in mind. I am not happy with automatic creation of a temporary branch, but it seems more “obvious” to the user than directly paralleling fetch.

  31. If you are not the remote user, then mucking about with the remote checkout is Bad Idea #1™. But the mirror of a fetch hurts nobody. If you are the remote user, then you’re not at all surprised either way.

  32. How about post-editing the first line near the title that this problem was fixed in recent versions. I read all of this just to figure out this is non-issue any more.

  33. tbear, it’s not a non-issue. It’s a mitigated issue. It won’t bite you like it did because you are protected unless you override it, but I still feel it’s a fundamentally wrong verb for what it’s doing.

  34. I agree some things seems strange and not well thought-out, but they are.
    You just need to practice it more :D

  35. My solution has been to use a staging branch on any working copy that isn’t bare. For example, some of my live websites are serving staging branches (with .git directories forbidden, importantly).

    That way, when you push, it updates the master branch, but that master branch itself isn’t checked out, so it doesn’t matter, the staging branch is checked out, and remains unchanged.

  36. As tbear said, I would have appreciated a couple of line saying that this issue has at least been mitigated. Thanks.

  37. Sorry, that sounds grumpy, I’m just new to Git, after spending quite a while finding a good system for version control in my company and was quite disappointed after reading just the title. I’m glad I decided to read further. :)

  38. $ git –version
    git version 1.8.2

    $ git push ../foo
    Counting objects: 5, done.
    Writing objects: 100% (3/3), 255 bytes, done.
    Total 3 (delta 0), reused 0 (delta 0)
    remote: error: refusing to update checked out branch: refs/heads/master
    remote: error: By default, updating the current branch in a non-bare repository
    remote: error: is denied, because it will make the index and work tree inconsistent
    remote: error: with what you pushed, and will require ‘git reset –hard’ to match
    remote: error: the work tree to HEAD.
    remote: error:
    remote: error: You can set ‘receive.denyCurrentBranch’ configuration variable to
    remote: error: ‘ignore’ or ‘warn’ in the remote repository to allow pushing into
    remote: error: its current branch; however, this is not recommended unless you
    remote: error: arranged to update its work tree to match what you pushed in some
    remote: error: other way.
    remote: error:
    remote: error: To squelch this message and still keep the default behaviour, set
    remote: error: ‘receive.denyCurrentBranch’ configuration variable to ‘refuse’.
    To ../foo
    ! [remote rejected] HEAD -> master (branch is currently checked out)
    error: failed to push some refs to ‘../foo’

  39. Yup, we’ve come a long way since 2008.


Leave a comment