The difference between Public Data and Open Data is in its release:
Open Data is data that is released with expressed re-use ability baked in. The data is released in an open data format, by an (ideally) authoritative source with a license attached that allows for re-use/remix of the data by anyone, without restrictions.
Public data is data that is released to the public that has some form of restriction(s) placed on it. These restrictions typically come from proprietary sources: data released in proprietary data formats, time sensitive releases, licensing from proprietary means used in creation, accessing, and re-using/re-mixing the data, etc.
PDFs are the textbook example of what is Public Data: data that has been released to the public, but includes barriers to access, editing, and re-use/re-mix. Even though PDF has been released by Adobe, its still tied to a contentious release that includes some technologies that are still owned and not shared by Adobe. While it is possible to create machine-readable PDFs, not to mention PDFs with attachments and accessible PDFs, I have yet to see two of these in the wild, and neither have you. The one that I have seen, was for a presentation by Adobe, and has since yet to be replicated, at least at a basic level of consumption whereas it would be noticeable.
The reluctance of Openness communities to define and own these terms makes them much more explosive the longer they are allowed to fester alongside each other. Without definition, there is no structure.