I was basically doing the same thing but I recorded what I just finished instead of what I was working on. I got that idea from gtimelog. I haven't been using it lately but I might go back to it.

    dun started
    dun Implemented feature xxx
    dun reddit and hn **
    dun Fixed bug xxx
    dun lunch **
    

Without an argument it just opened the log file in vim.

I like this style of recording things but I would like to be able to log back to a server from anywhere and have things like git commits added as well for additional context. And then there is report generation.