Yes, just looking at child-porn is illegal, as is downloading any. And the government's definition of "downloading" is anything that actually reached your computer screen.
Oddly enough, that definition actually protects the casual surfer to a certain extent. The prosecution lawyers are so enamoured of that definition that they won't risk a serious legal challenge that might scare the courts into tightening it up, to, say, deliberately sought out pictures that reach your computer.
So, if you accidentally linked to a kiddie-porn site (and yeah, it happened to me once too) you should be fine; the prosecutors have a) better things to do and b) a vested interest in not abusing the situation.
Of course, if you have a history of accessing such sites...