go to ORKG: http://orkg.org/orkg/predicate/P103000

pretraining architecture

The underlying neural network design used during pretraining (e.g., transformer-based, RNN, CNN).