For any prime $p>5$ we construct a Calabi-Yau threefold $X$ defined over a finite extension $K$ of $\mathbb{Q}_p$ such that every model of $X$ over $\operatorname{Spec}\mathcal{O}_K$ has singular special fiber, yet the Galois action on the $\ell$-adic cohomology group $H^3_{\acute{e}t}(X,\mathbb{Q}_\ell)$ is unramified for $\ell\neq p$ and crystalline for $\ell=p$. This provides a counterexample to the analogue of the N\'eron-Ogg-Shafarevich criterion for three-dimensional Calabi-Yau manifolds.