Pauli Kilpiä played amateur pesäpallo for many years, winning league championships in 1950 and 1957 with Kuusankosken Puhti. In 1952 he was selected to represent the Workers’ Athletic Federation team in the demonstration event at that year’s Helsinki Olympics, which they lost 8-4. He continued to play the sport into the 1970s and also took part in amateur ice hockey. In 2012, when Finland celebrated the 60th anniversary of the Helsinki demonstration event, he was one of the surviving players honoured before the start of the commemorative match. At the time of his death, he was the last surviving member of his team and the penultimate survivor of the event overall.