It's fine as long as the graph of references is immutable, or is unidirectional/acyclic with unique ownership.
If you have backreferences or "parent pointers", you need `Arc<Mutex<...>>` or `Rc<RefCell<...>>`, and then you run into trouble as you encounter the same node multiple times while traversing the graph, because you cannot hold a mutex lock or mutably borrow `RefCell` twice in the same call stack.
The solution with much less resistance in Rust is to go for a data-oriented representation. If you really need an actual graph of objects, separate the node data from the topology metadata, and refer to the nodes using an ID or index. (As an extra bonus, this also gives you much better cache locality.)