A dynamical system is an evolving "system" in its broadest meaning. While a monogenic system is a specific class of systems in classical mechanics.
From the OP's wikilinks (my emphasis):
a dynamical system is a system in which a function describes the time dependence of a point in an ambient space. Examples include the mathematical models that describe [...] pendulum [...] water [...] particles in the air [...] number of fish
In classical mechanics, a physical system is termed a monogenic system if [...] all forces, with the exception of the constraint forces, are derivable from the generalized scalar potential, and this generalized scalar potential is a function of generalized coordinates, generalized velocities, or time
Dynamical systems theory is a area of mathematics which concerns itself with the qualitative behavior of any evolving mathematical system; it can be informally called nonlinear dynamics when done by physicists: who often won't bother to restrict their areas of research to the traditional branches of physics, and certainly not to mechanical systems only.