Hopefully your captions will stay anchored, but you never know how some ebook devices will interpret everything.
You may want to place captions under the photos before Inserting images to Word. I use CorelDraw for such purposes, but there are other programs which will do the same thing. This way you won't have to worry how your ebook is rendered.
In CorelDraw it is a simple matter of placing text where/how I want it under an image then using mouse to group both text and image before exporting. In CorelDraw there is a box to tick indicating you want to Export only the selected items. Other programs may operate differently so follow instructions provided by your particular program.
I have images throughout the non-fiction books I have published. I have had no real problems with captions staying where they should as long as the image itself is anchored. Ted's suggestion may offer more certainty, though.