I think if something is discovered in undisturbed rock or sediment layer that then it a safe assumption that the object was there at the time the layer or rock was formed. The people who found these objects could have simply been mistaken or possibly made up the whole thing.

That said, the Antikythera Mechanism and Baghdad Battery are very real artifacts that have been well studied and even reconstructed.