Herman de By became acquainted with sports while attending a British public school in Ramsgate. Starting as a tennis player, he switched to swimming, and earned a spot in the Dutch swimming team that was sent to the Paris Olympics, where he competed in the 200 m freestyle, but failed to make the final. Other known results include a 2nd place in the 1897 (Open) Belgian Championships (100 m freestyle). The Rotterdam native then turned to boxing, which he took very seriously. He trained in particular for the 1908 London Games, even engaging two British coaches, but a broken wrist shortly before the competitions forced him to withdraw. A third sport in which de By excelled was rowing. Already participating in the Dutch Championships around the turn of the century, De By continued to row well into his 80s. In an interview a few weeks before his death, the then 87-year-old was shown to still have a competitive spirit, rowing for speed rather than leisure.