sci.op-research

On Feb 14, 1:12 pm, "Bob Daniel"
wrote:
> >> > Let me return to the original problem which is pre-processing, not
> >> > solver management during optimization. I happen to use MPL as my
> >> > model manager. But the paradigm I've increasingly followed is
> >> > separating data management from model management as much as I can. If
> >> > you are into a data base to service the model for a penny, you are in
> >> > for a pound. So you might as well leverage it's full capabilities
> >> > because it's a lot easier to manipulate data on that platform than it
> >> > is with GAMS or MPL.
>
> >> > My models can be intermediate sized, and when I need to support one
> >> > with a data base, I use Access. I can generate almost every input
> >> > vector in Access using the Query Builder including index sets and pass
> >> > those such that MPL merely reads in the data/indexes and passes those
> >> > on to the with very little manipulation formulation. And error
> >> > checking is much easier this way.
>
> ...
> >> > SteveM
>
> >> But how would your paradigm cope when you have data coming in from
> >> databases
> >> of various types possibly scattered across many sites?
> >> Bob Daniel
>
> > Bob,
>
> > Again, I don't run large scale models so do not have that
> > requirement. When I was in the OR group at American Airlines long
> > ago, we used SAS for data management of distributed legacy data bases
> > until we got the model up and running. And then we handed data
> > management over to the Sabre programmers. But those were also the
> > MPSX days where we had to write our own MPS file generators.
>
> > But with a query builder like Access, it's much easier for me to see
> > incrementally how queries are building out. I mean wham, bam, back
> > and forth testing queries with sample data sets. MPL is much less
> > transparent in doing that.
>
> > You are obviously much more experienced on that end. If you want to
> > give us an idea of your approach for data management given current
> > software capabilities, that would be great. Can you follow this with
> > something brief?
>
> > Thanks,
>
> > SteveM
>
> I'm not sure I can be brief without being facile. I'll try to make it not
> sound like a commercial.
>
> When we did Xpress' lp-model it was almost de rigeur to stick to the mantra
> that separation of model and data was a Good Thing. We would have been
> slaughtered by the academics had we not done this. But when we decided to
> replace lp-model a few years ago we had long and hard thoughts about what
> paradigm to adopt and talked to lots of our users.
>
> Xpress-Mosel was the result of this, and it's radically different in that
> it's a programming language with math programming things as first class
> objects. We decided on this approach as lots of users wanted the ability to
> do fairly rigorous data manipulation, comparisons and checking in the same
> place as where the model was defined. Data coming from more than one
> database, which is inherent in modelling using on-line data from Production,
> Marketing, Sales, Inventory etc are a fact of life. And IMHO these have to
> be reconciled at one central location in many models.
>
> So Mosel had to have full programming capabilities. OK, yet another language
> to learn but it's pretty easy (even an old FORTRAN man like me could pick it
> up!) The programming aspect has other benefits we wanted too, like being
> able to write complex optimization schemes e.g. decomposition, and be
> extensible. Since it's compiled one can hide the intellectual property/
> prevent naive users messing up the model.
>
> Obviously the modeller has to have the ability to suck raw data out of, and
> pump answers back to, external databases, so that was a must have. In the
> old days a Report Writer was the final leg of the tripod, with modeller and
> optimizer. Having a programming capability renders that obsolete.
>
> I certainly wouldn't decry doing lots of stuff in Access/whatever. If one is
> familiar with that then one can clearly be highly productive. As long as the
> whatever isn't Excel ...
>
> Bob
> These are my personal opinions.

Bob,

Thanks very much for the insights. If I stumble across something
substantial, now I know where to turn.

Incidentally, re: your Excel comments. I like most people use Excel
and I develop relatively simple models in Excel too using the MPL
object library Optimax. But since I'm writing VB code, I visit the
Excel VBA newsgroup to post questions. And you have no idea have many
people visit that site deperately looking for VBA ways to manage
unnormalized data bases with Excel. They often post these gigantic
blocks of code to implement relative simple data base manipulations.

I've actually responded to some with the suggestion that they consider
moving the model to Access or a .Net solution. That always sinks like
a rock. Maybe it's being an OR guy, but that kind of illogic
perplexes me...

SteveM