2
$\begingroup$

I've been thinking for some time about how to better find the optimal weights for neural networks and it struck me that when solving Lagrangian mechanics problem you are optimizing the action function.

Thus my question became: Is it possible to set up a physical system (Circuit or otherwise) such that the action function for the system represents/"is the same as" the mathematical function you want to minimize. Thus you could physically measure the system and get the optimal solution for your mathematical function, regardless if it's non-linear or convex etc.

I'm sure this must have been done somewhere, I've been searching around forever trying to find people doing this but I can't find anything. Does anyone know what this field or research is called or if it's even possible?

EDIT: I'm not looking for maths of how to optimize problems, I'm looking to build a Physical real world system which when run can be measured and those measurements would correspond to the optimal point in the mathematical system. Sort of like "Hardware accelerating" optimization problems.

Improve this question
$\endgroup$
2
  • $\begingroup$ Yes, indeed, this is a very old idea. An action functional is just one specific example of a functional, and a functional is just a function of a function. Finding extrema is done in exactly the same way. I am not knowledgeable in the area of neural nets, but the mathematics is exactly the same: once you fix the functional, its Euler-Lagrange equations give you candidates for extrema. Once you linearlize the Euler-Lagrange equations, you can do a stability analysis to determine whether it is a local extremum or a saddle point. $\endgroup$
    – Max Lein
    May 20, 2019 at 2:03
  • $\begingroup$ I don't quite think you understood my question, or I misunderstood your answer. I'm not looking for the maths behind optimization, I'm looking to build a physical circuit/system to do the optimization for me. In fact I'll update it to make it clearer. $\endgroup$
    – Beacon of Wierd
    May 21, 2019 at 0:52

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.