I came to optimization from symbolic AI, and I work between the two. What holds
them together is a single idea: the difficulty of a problem lies in how it is
written, so the useful move is to rewrite it — as a set of smaller problems,
or in a different formalism altogether.
Symbolic AI. Knowledge representation and reasoning, which is where I
started: rule-based expert systems for domains whose knowledge is codified and
whose reasoning has to stay inspectable, from the diagnostic criteria of the
DSM-5-TR encoded in Prolog to fuzzy rules in CLIPS, and lexical semantics over
WordNet. Above all, translation between formalisms: a planning problem can be
stated as a constraint satisfaction problem and, from there, as a propositional
formula, at which point a CDCL solver applies to it. I implemented that chain of
translators, and the CDCL solver itself, in
aima-python.
Numerical optimization. The same move with a different alphabet. Lagrangian
relaxation and Benders decomposition, the bundle methods that solve the
nondifferentiable duals they give rise to, and the parallel solution of the
subproblems, which a decomposition produces by the hundred and which are
independent by construction. This is the premise of
SMS++, where a model is declared as a tree
of Blocks: the same instance can then be solved
monolithically, dualized along its coupling constraints, or split by scenario, and
which of the three happens is a matter of configuration rather than of rewriting
the model. Reformulation returns here as recognising, inside a subproblem, a
structure that already has an efficient algorithm: the subproblems of a capacitated
facility location are binary knapsacks, a single-unit commitment is a shortest path
on a run-length graph, a dispatch is a min-cost flow.
Connecting the two. The reason I keep a foot on each side. Heuristics learned
from data, placed exactly where an exact method makes a choice — the branching
of a SAT solver in
NeuroSAT, the step size of a bundle
method — so that the learned part chooses and the exact part still guarantees.
And symmetrically, learning problems written as structured optimization models: a
support vector machine is a Block like any other, and the
same decomposition machinery applies to it.