Apparently the direct source is a little unclear, but I just found a good lead:
http://www.wordorigins.org/wordork.htm
"The New York Times, on 24 December 1946, credited James J. Kilroy of Quincy, Massachusetts with starting the craze. Kilroy was an inspector at the Bethlehem Steel shipyard in that city, and used a yellow crayon to write Kilroy was here on items that he had inspected. The graffiti became a common sight around the shipyard, and was imitated by many of the other 14,000 shipyard workers when they were drafted and sent around the world."
I'm going to try and track down a copy of that NY Times article. I'll also start posting some of my screen caps here too.