AbstractModel, ConcreteModel, DataPortal and Problem dumps with Pyomo
And flex your muscles!
In the previous article we solved our first problem with Pyomo and SCIP. Today we solve a second problem. We introduce the concepts of AbstractModel, ConcreteModel and DataPortal. We also learn how to export a Problem into a text file for debugging.
The Truck Allocation Problem
We run a generalist product supplier business that sells a variety of products worldwide. Every day a truck comes to the warehouse to load the items that must be delivered to retailers. Usually, the number total number of items to be shipped exceed truck's capacity. Therefore we must decide which items we load into the truck and which items stay on the warehouse until the next truck.
Retailers do not pay in advance, meaning that we only get paid for the products that we actually send to them. Hence, we want to maximize the profits of all items that we ship with the truck.
As we already mentioned in the first article, a well defined Mathematical Programming Problem packs all the information that we need to understand and solve this problem. The following Pyomo implementation should be clear enough.
doc, docstrings, variable names and comments are crucial to understand the code and the Problem itself. They are the lighthouse for code readers and maintainers. Stick this in your mind.from pyomo.environ import AbstractModel
model = AbstractModel(name='truck')
Sets and Parameters
from pyomo.environ import (
NonNegativeIntegers,
NonNegativeReals,
Param,
Set,
)
model.products = Set(
name='products',
doc='Set of products to load into the truck.',
)
model.stock = Param(
model.products, # One parameter value for each element in products set
name='stock',
doc='Total number of items requested of each product',
domain=NonNegativeIntegers,
)
model.profits = Param(
model.products, # One parameter value for each element in products set
name='profits',
doc=(
'Profits obtained for each item that we load into the truck. '
'Units are €/item'
),
domain=NonNegativeReals,
)
model.volume = Param(
model.products, # One parameter value for each element in products set
name='volume',
doc='Volume of each product in cubic meters',
domain=NonNegativeReals,
)
model.truck_volume = Param( # Just one parameter
name='truck_volume',
doc='The volume of the truck in cubic meters',
domain=NonNegativeReals,
)
Variables
from typing import Tuple
from pyomo.environ import AbstractModel, NonNegativeIntegers, Var
def number_of_items_bounds(
model: AbstractModel,
product: str,
) -> Tuple[int, int]:
"""Get the lower bound and upper bound of variable stock."""
lower_bound = 0
upper_bound = model.stock[product]
return lower_bound, upper_bound
model.number_of_items = Var(
model.products, # One variable for each element in products set
name='number_of_items',
doc='Number of items to load into the truck',
domain=NonNegativeIntegers,
bounds=number_of_items_bounds,
)
We define a variable that takes non negative nteger values (0, 1, 2, ...). The number of items to ship is upper bounded by the total number of items to be sent.
The argument bounds only admits a callable that return two floats. If you need to bound a variable with a Pyomo expression, you must use a constraint.
Constraints
from typing import Tuple
from pyomo.environ import AbstractModel, Constraint
from pyomo.core.expr.relational_expr import InequalityExpression
def constraint_volume_of_items_cannot_exceed_truck_volume(
model: AbstractModel,
) -> InequalityExpression:
"""The volume of items in the container cannot exceed truck volume."""
shipped_volume = sum(
model.number_of_items[product] * model.volume[product]
for product in model.products
)
return shipped_volume <= model.truck_volume
# Only one constraint.
model.volume_of_items_cannot_exceed_truck_volume = Constraint(
name='volume_of_items_cannot_exceed_truck_volume',
doc=constraint_volume_of_items_cannot_exceed_truck_volume.__doc__,
rule=constraint_volume_of_items_cannot_exceed_truck_volume,
)
Objective
Thanks to the function name, the objective is crystal clear. We want to maximize the shipped_profits.
from pyomo.environ import AbstractModel, Objective, maximize
from pyomo.core.expr.numeric_expr import LinearExpression
def shipped_profits(model: AbstractModel) -> LinearExpression:
"""Sum the profits of all items that we load into the truck."""
shipped_profits = sum(
model.number_of_items[product] * model.profits[product]
for product in model.products
)
return shipped_profits
model.objective_function = Objective(rule=shipped_profits ,sense=maximize)
Instantiate a ConcreteModel
Mathematics build algorithms that solve Abstract Problems theoretically. This problem we're solving here is known as the Knapsack Problem, and it is already solved mathematically for any set and parameters that you choose.
In Pyomo, an AbstractModel is a template with no data. Still the values of the sets and parameters are not defined. The bounds of the variables and the actual expression of the constraint is not defined neither. Thus, a Solver cannot solve instances of AbstractModel because they have no data.
The Solver works with instances of ConcreteModel. Think of a ConcreteModel an AbstractModel (the templated) filled with actual data. Eventually, the Solver also stores the optimal solution (and other non-optimal solutions as well) into the ConcreteModel.
One way to load data with Pyomo is by using a DataPortal. In this example the DataPortal loads the file data.json containing the values of Sets and parameters. Here's the file.
# File: data.json
{
"products": ["toys", "treadmill", "locker", "costumes"],
"stock": {
"toys": 15,
"treadmill": 10,
"locker": 5,
"costumes": 20
},
"profits": {
"toys": 12,
"treadmill": 95.0,
"locker": 200,
"costumes": 40.0
},
"volume": {
"toys": 0.5,
"treadmill": 3.4,
"locker": 7.2,
"costumes": 1.4
},
"truck_volume": 50.0
}
Data seems (and is) unrealistic. Bear with that. It has been crafted specifically to generate an interesting discussion later on.
from typing import Tuple
from pyomo.environ import ConcreteModel, DataPortal, SolverFactor
# Load data from json file. Pyomo admits loading data in
# many formats. Here are more examples
# https://pyomo.readthedocs.io/en/stable/working_abstractmodels/data/dataportals.html
data = DataPortal(filename='data.json')
# Get an instance of the problem with data.
instance: ConcreteModel = model.create_instance(
name='Small container',
data=data,
)
# Get the SCIP solver
solver = SolverFactory('scip')
# Finally, solve the problem!
solution = solver.solve(
instance,
tee=True, # Log the progress of the execution of the Solver
)
Notice that the separation of AbstractModel and ConcreteModel is really convenient. We can define just one AbstractModel, but solve many instances ConcreteModel instances with different data each.
Report results
from pyomo.environ import value
profits = value(instance.objective_function)
print(f'Profits are {profits}€')
for product in instance.products:
number_of_items = round(instance.number_of_items[product].value)
volume_of_an_item = instance.volume[product]
volume_of_items = volume_of_an_item * number_of_items
print(
f'Put {number_of_items} units of {product!r}. '
f'This occupies {volume_of_items} cubic meters'
)
Notice that we read results from the ConcreteModelinstance. Try to execute profits = value(model.objective_function) and Pyomo will warn you that model does not hold any data, so no value can be computed.
The output of the Solver
We'll use the same environment we installed in the previous article.
Run this file. The solver shows logs of the whole process to help us understand the complexity of the problem and the execution process. For now, let's focus on the solution of the problem. In subsequent articles we'll understand logs in detail.
Summary of the problem
Near the beginning there's a summary of our problem.
original problem has 4 variables (0 bin, 4 int, 0 impl, 0 cont) and 1 constraints
The problem has 4 integer variables, one for each product, and just one constraint.
For completeness, bin refers to binary (as we saw with the Rooks Problem), cont refers to continuous and impl refers to implications (haven't seen any examples yet).
The optimal solution
Near the end there's the solver veredict. It found 12 solutions in 0.00 seconds. One of them is the optimal solution.
SCIP Status : problem is solved [optimal solution found]
Solving Time (sec) : 0.00
<line omited>
Primal Bound : +1.41000000000000e+03 (12 solutions)
<lines omited>
Profits are 1410.0€
Load 0 unit of 'toys'. This occupies 0.0 cubic meters
Load 6 unit of 'treadmill'. This occupies 20.4 cubic meters
Load 1 unit of 'locker'. This occupies 7.2 cubic meters
Load 16 unit of 'costumes'. This occupies 22.4 cubic meters
Few things to consider:
It found 12 solutions. This means, 12 valid item arrangements in the truck. Finally, the solver asserts that (at least) one of them is the optimal solution to the Problem.
Notice that we don't reach the
stocklimit of any product. Also, the volume of all items that we ship (50) does not exceed the volume of the truck (50). The Solver did a great job at filling every corner of the truck.The
lockeris the product with higher profits per item. However, we're not shipping all units that we can, but just one.💡Can you think why this happens?💡Try adding a constraint so that the number of lockers to load into the truck is two. The new solution won't as good, meaning that the profits will be lower or equal!
Debugging our implementation
With larger and more complex AbstractModel implementations we may need to double check if we typed our Sets, parameters, variables and constraints are correctly. We can export a ConcreteModel into a text file. Use it often, it is life-saver.
# If the instance has been already solved, the dump also contains
# the values of the optimal solution.
with open('concrete_model_dump.txt', 'wt') as f:
instance.pprint(f)
Here's the file
doc arguments and docstrings are!1 Set Declarations
products : Set of products to load into the truck.
Size=1, Index=None, Ordered=Insertion
Key : Dimen : Domain : Size : Members
None : 1 : Any : 4 : {'toys', 'treadmill', 'locker', 'costumes'}
4 Param Declarations
profits : Profits obtained for each item that we load into the truck. Units are €/item
Size=4, Index=products, Domain=NonNegativeReals, Default=None, Mutable=False
Key : Value
costumes : 40.0
locker : 200
toys : 12
treadmill : 95.0
stock : Total number of items requested of each product
Size=4, Index=products, Domain=NonNegativeIntegers, Default=None, Mutable=False
Key : Value
costumes : 20
locker : 5
toys : 15
treadmill : 10
truck_volume : The volume of the truck in cubic meters
Size=1, Index=None, Domain=NonNegativeReals, Default=None, Mutable=False
Key : Value
None : 50.0
volume : Volume of each product in cubic meters
Size=4, Index=products, Domain=NonNegativeReals, Default=None, Mutable=False
Key : Value
costumes : 1.4
locker : 7.2
toys : 0.5
treadmill : 3.4
1 Var Declarations
number_of_items : Number of items to load into the truck
Size=4, Index=products
Key : Lower : Value : Upper : Fixed : Stale : Domain
costumes : 0 : 16.0 : 20 : False : False : NonNegativeIntegers
locker : 0 : 1.0000000000000053 : 5 : False : False : NonNegativeIntegers
toys : 0 : 0.0 : 15 : False : False : NonNegativeIntegers
treadmill : 0 : 5.999999999999989 : 10 : False : False : NonNegativeIntegers
1 Objective Declarations
objective_function : Size=1, Index=None, Active=True
Key : Active : Sense : Expression
None : True : maximize : 12*number_of_items[toys] + 95.0*number_of_items[treadmill] + 200*number_of_items[locker] + 40.0*number_of_items[costumes]
1 Constraint Declarations
volume_of_items_cannot_exceed_truck_volume : The volume of items in the container cannot exceed truck volume.
Size=1, Index=None, Active=True
Key : Lower : Body : Upper : Active
None : -Inf : 0.5*number_of_items[toys] + 3.4*number_of_items[treadmill] + 7.2*number_of_items[locker] + 1.4*number_of_items[costumes] : 50.0 : True
8 Declarations: products stock profits volume truck_volume number_of_items volume_of_items_cannot_exceed_truck_volume objective_function
This dump also offers valuable information. For instance, notice that the column Body of constraint volume_of_items_cannot_exceed_truck_volume computes the volume of all the items that we load into the truck. We can use Pyomo to get the expression evaluated with the optimal solution.
from pyomo.environ import value
volume_of_all_items_loaded_into_the_truck = (
value(instance.volume_of_items_cannot_exceed_truck_volume.body)
)