We wait for chickens and ducks to lay eggs before they are collected so they are still alive when their eggs are collected, and yes the eggs themselves are generally unfertilised so they can never become baby birds.
Birds may be kept in battery cages, and may be killed when they get too old to produce eggs in the same quantities as when they were young, so not all vegetarians do eat eggs. Many asian vegetarians would not consider the eating of eggs to be vegetarian at all.
Caviar however was traditionally obtained by killing the fish and extracting the eggs from its dead body and byproducts of slaughter have always been non-vegetarian.
These days caviar is often produced either by removing the ovaries of the fish together with the caviar through an incision or by cutting into the fish in caesarian section, removing the eggs and then stitching the fish up afterwards. Although these processes do leave the fish alive they can hardly be said to be unharmed.
Some caviar is however produced by massaging the fish, although that is still achieved by making an incision into it.
In the end you need to make your own decision as to whether you're comfortable with any of the processes involved in obtaining any kind of eggs.
Are the fish eggs fertilised, unlike most chicken eggs
... why would a chicken lay an unfertilized egg?