Maybe someone could find something in emigration papers.

I thought that if his brother stayed in Sweden, he and children of him might be find-able in 1880 and 1890 "Census". No Parker in (the very incomplete) 1880 census, three in 1890. On born in 1876 in Great Britain, one born in India in 1840. Left is one Johan Alfred Parker Johansson, born 1848 in Ytterby, Göteborgs och Bohus län. There's two Johan Alfred/Alfrid with father Johan/Johannes born in Ytterby in 1848, but both seem to have "plain" Swedish parents, so it doesn't seem to exist any brother to Charles/Carl in the late 19th Century Census records.

Maybe Charles took the name Parker upon immigration, maybe he "jumped ship" and didn't want to be found, making up most of the story...?