Dynamic Scheduling of Shuttle Robots in the ... - ACS Publications

Article pubs.acs.org/IECR

Dynamic Scheduling of Shuttle Robots in the Warehouse of a Polymer Plant Based on Dynamically Configured Timed Automata Models Christian Schoppmeyer,*,† Subanatarajan Subbiah,§ Josè Manuel De La Fuente Valdès,‡ and Sebastian Engell† †

Process Dynamics and Operations Group, Department of Biochemical and Chemical Engineering, TU Dortmund University, 44227 Dortmund, Germany § ABB Corporate Research Center, 68526 Ladenburg, Germany ‡ M+W Process Automation, 71088 Holzgerlingen, Germany ABSTRACT: The increasing demand for customized products and the pressure to produce them on time with low-cost upon customers’ demands not only call for efficient and economical production processes but also require high-performance logistic planning to make the best possible use of the available resources. The elements to be planned besides the production itself include preproduction material handling, such as raw material deliveries and storage, and the postproduction material handling, such as storage and distribution of products. Specialty products are usually stored in warehousing systems where the speed of the removal and the distribution of products can strongly affect the efficiency of the complete system. Hence, optimal scheduling of the distribution operations is required to meet throughput targets and lead to improved efficiency and financial gains. In this contribution, we deal with a bottleneck subsystem of an industrial polymer processing plant, the pallet removal system of a high-rise warehouse. A dynamic scheduling approach that uses optimization to solve the problem on a moving window is proposed. The shuttle management problem is modeled using timed automata and is solved by cost-optimal reachability analysis. The most challenging aspect of the problem is introduced by the a priori unknown arrival times of the pallets and their unknown routes. The proposed dynamic solution approach is compared to an enhanced version of the currently used first-come, first-served dispatching rule. The comparative study shows that the proposed approach leads to makespan reductions of about 30%.

1. INTRODUCTION The efficiency of a flexible manufacturing system strongly depends on the effective usage of the limited set of available resources to fulfill the customer orders cost-efficiently and according to the promised delivery dates. This is addressed by planning and scheduling, where planning is understood as acting on long to medium horizons while scheduling is concerned with short-term decisions on which resources to use when, in order to perform the different production tasks. Scheduling is needed not only in the production unit itself, where usually, in the chemical industry, the durations of the operations are quite long, so that computing times in the order of minutes are acceptable, but also in other elements of the preand postproduction logistic processes where the durations of the individual operations are much shorter. Such an example is considered here: the management of the shuttle system in the warehouse of a polymer production plant. The shuttle system is a key link between the warehouse and the loading of pallets of material on waiting trucks. One of the crucial characteristics that make the assignment, sequencing, and timing decisions challenging is the presence of uncertainty in the information on the operational level (such as arrival times of the jobs, unit breakdowns, etc.). Highly dynamic production environments and manufacturing plants demand optimization techniques that can compute new decisions on the operational levels within small amounts of computation time in © XXXX American Chemical Society

order to react to unforeseen events. The problem we address here is representative for this situation: the shuttle management system only receives information about the next jobs it has to process in real-time over a limited time horizon. Thus, fast realtime capable solutions are needed. In the next sections, we discuss the state of the art in scheduling techniques that have been proposed for similar problems in the areas of chemical batch scheduling, production scheduling in manufacturing environments, and order scheduling in automated warehouse systems. 1.1. Chemical Batch Scheduling. Important contributions and reviews on approaches toward static batch scheduling have been reported in the works of Reklaitis,1 Mockus and Reklaits,2 Pekny and Reklaitis,3 Shah,4 Ierapetritou and Floudas,5−7 Kallrath,8,9 Lim and Karimi,10 Maravelias and Grossmann,11 Floudas and Lin,12 Castro et al.,13 Sundaramoorthy and Karimi,14 Shaik et al.,15 Mendez et al.,16 Zheng et al.,17 and Maravelias.18 Reklaitis1 lists the common sources of uncertainties in batch scheduling problems as follows: (a) arrival times of the jobs, (b) the availabilities of equipment, (c) the processing times, Special Issue: Jaime Cerdá Festschrift Received: January 30, 2014 Revised: May 30, 2014 Accepted: August 29, 2014

A

dx.doi.org/10.1021/ie500437r | Ind. Eng. Chem. Res. XXXX, XXX, XXX−XXX

Industrial & Engineering Chemistry Research

Article

and (d) off-spec products. Shah4 and Floudas and Lin12 present a comprehensive review on uncertainties in production scheduling problems and classify the approaches to solve them into two main categories: reactive and stochastic approaches. In the reactive approaches, the uncertainties are considered implicitly such that, for the given information on the problem, a nominal schedule is computed initially and at the occurrence of an uncertain event a new schedule is recomputed and implemented. The reactive approaches are used mostly in problems with short-term uncertainties in the problem parameters, such as arrival of a new job with high priority, failure of processing units, etc. In contrast, the stochastic scheduling approach is traditionally used to handle longer-term uncertainties, such as customer demands, etc. For details, the reader is referred to the literature.19−23 Many contributions in the area of reactive scheduling have been made in the past decades. Kanakamedala et al.24 proposed a procedure to handle uncertainties in the availabilities of the processing units and in the processing times by exploring all possible reroutes of the product in the case of an uncertainty and choosing the reroute decision that has least impact. Vin and Ierapetritou25 proposed a continuous time MILP formulation based on a two-stage solution approach to handle problems with uncertainties, in particular unit unavailability and rush order arrival. In the first stage, a deterministic schedule is established based on the current data, and at the second stage all possible rescheduling alternatives are incorporated and a reoptimization is performed to obtain a new schedule. Mendez and Cerda26,27 proposed a continuous time based MILP formulation which provides multiple rescheduling actions by rescheduling the operations involving local reordering of the tasks by reallocating the units and by inserting new batches. The unforeseen events handled by the approach include deviations in processing times, equipment breakdown, or when the product produced had to be reproduced due to violations of the specifications. Van den Heever and Grossmann28 proposed a multiperiod MINLP problem formulation to integrate production planning and reactive scheduling in a two level approach where the first level solves the planning problem and the lower level solves the scheduling problem. Janak et al.29 presented a MILP formulation to handle equipment breakdowns or arrivals of new and high-priority orders. In contrast to many scheduling approaches presented using MILP, the contribution from Janak et al.29 presents the idea to avoid full rescheduling of the current production horizon by detecting the tasks which are not affected by the unforeseen event. The resulting tasks along with additional subsets of tasks are then fixed in the MILP problem, and the rest of the jobs are rescheduled. Alternatives to equation-based modeling of scheduling problems and solving mathematical programs are algorithms developed based on experienced guesses or based on rules of thumb, commonly termed as heuristics, and approaches inspired from biological evolution (such as evolutionary algorithms, genetic algorithms, and ant colony optimization) which are commonly termed as meta-heuristics, e.g. Capon-Garcia et al.30 Recently, approaches that combine the strengths of mathematical programming and meta-heuristics, known as math-heuristics, have also been used to solve scheduling problems (Della Corce et al.31). Besides the above-mentioned techniques, approaches based on graph-theory have also been used to solve combinatorial problems in process scheduling (Hegyhati and Friedler32). 1.2. Production Scheduling in Manufacturing Environments. Based on the standard three-field classification

scheme of Graham et al.,33 many static scheduling problems and the corresponding solution approaches based on mathematical programming models, genetic algorithms, dispatching rules, and many more for manufacturing facilities have been reported and reviewed (Pinedo,34 Lozano and Medaglia,35 Zhu and Wilhelm,36 Lin et al.,37 Herrmann et al.38). According to Vieira et al.,39 rescheduling is practically mandatory in dynamic, stochastic manufacturing environments to minimize the effect of unexpected events disturbing the existing agenda. Common types of disturbances are identified as machine failures, processing time delays, rush orders, quality problems, and unavailable material. The authors in Vieira et al.39 present a classification scheme for rescheduling approaches for manufacturing systems based on the rescheduling environment, the rescheduling strategy, and the rescheduling method for schedule generation and schedule repair. The rescheduling environments are distinguished between static environments considering a finite set of jobs, e.g. used in cyclic scheduling that is based on deterministic job arrival data or robust scheduling, and dynamic environments with an unknown infinite set of jobs, e.g. environments with unknown job arrivals and job routes. The rescheduling strategies are classified in dynamic strategies which are continuously executed, e.g. dispatching rules, and predictive-reactive strategies which are either executed periodically, are event-driven, or are in a hybrid fashion. The periodic and hybrid strategies are also known under the term rolling time horizon or moving window approach. The rescheduling methods are classified according to the schedule generation, either nominal based only on currently available and deterministic information, or robust explicitly considering potential disruptions in the future execution, and according to the schedule repair strategy: right-shift scheduling, partial rescheduling, and complete regeneration. A detailed discussion of robust (proactive) approaches with focus on robustness and stability measures is presented in Sabuncuoglu and Goren.40 Many contributions in the area of dynamic scheduling in manufacturing environments have been made in the past decades (Herrmann,41 Ouelhadj and Petrovic42). Church and Uzsoy43 discuss the impact of periodic and event-driven rescheduling policies for a single-stage problem with known release and due dates based on the earliest-due-date dispatching heuristic. Muhlemann et al.44 evaluate different dispatching heuristics for the dynamic job shop problem with unknown order (job) arrival times, resource (machine) breakdowns, and operation (task) delays. Bierwirth and Mattfeld45 propose a genetic algorithm with a tailored problem decoding to solve static, dynamic, and nondeterministic job shop problems. Lin et al.46 combine a genetic algorithm with particle swarm optimization to an hybrid evolutionary algorithm to solve dynamic job shop problems with resource breakdown and due date modifications. The particle swarm optimization is integrated into the genetic algorithm to generated a new chromosome based on the pool of genes of the current generation. In Nie et al.,47 a gene expression programming-based approach to generate reactive scheduling policies for dynamic flexible job shop problems with unknown order arrival times is developed for different objective functions. Gomes et al.48 propose a reactive scheduling approach based on a continuous time mathematical programming formulation in which the schedule repair strategy can be varied from right-shift rescheduling to complete regeneration for a flexible job shop problem with order reentrance and unknown order arrival times. In Kopanos B



Article

and Pistikopoulus,49 a MILP-formulation is extended to a multiparametric programming formulation and embedded into a moving window approach for scheduling of a network of combined heat and power units with unknown but bounded heat and electricity demands. 1.3. Order Scheduling in Automated Warehouse Systems. Following the definition of Rouwenhorst et al.,50 the short-term decisions on the operational level of warehouse systems can be classified into decisions concerning the assignment of trucks to docks in the receiving and the shipping processes, decisions of the storage process−assignment of replenishment tasks and allocation of products to free storage locations, and decisions in the order picking process related to batch formation, assignment, and sequencing of picking tasks to order pickers and selection of dwell points for idle order picker equipment. Of the mentioned decision areas, the process of order picking has been identified as the most labor-intensive process when manually operated and as the most capitalintensive process when automated systems are used (de Koster et al.51). For larger warehouses where automated systems are implemented, the order picking (and order storing using the same device) is usually realized by an automated storage/ retrieval system (AS/RS) equipped with separate-rail stacker cranes which fetch the items or pallets from the racks and transport the load to the shipping area (Haneyah et al.52). To operate efficiently, the order picking process needs to be optimally controlled, especially in the case when the picking jobs depend on unknown truck arrival times and decisions have to be made in real-time. Amato et al.53 proposed a control algorithm for the management of an automated warehouse system which optimizes the storing and picking operations of the stacker cranes and the movement operations of a single shuttle further distributing the loads fetched by the cranes in the shipping area. The optimization algorithm for the storing and picking operations is a dispatching rule working on a static set of operations that selects for each storing operation of the set a succeeding picking operation that has the minimum travel time between the two locations in the rack. The optimization algorithm for the movement operations of the shuttle is a second dispatching rule that selects pairs of picking and dropping operations according to minimum traveling distances from the current position to the picking location and to the corresponding dropping location. Both optimization algorithms are integrated into a hierarchical management system which itself is modeled together with the actual warehouse in colored timed Petri nets. The scheduling in the hierarchical management system, in detail the allocation of sets of storing and picking operations to the cranes, is done on a so-called management system level by a dispatching rule. The authors of ref 54 propose a dynamic programming approach to schedule clusters of storing and picking orders for multiple stacker cranes on common rails in an automated storage/retrieval system. An additional challenge is the possible interferences of the stacker cranes which operate on the same common rail, causing a detour or a wait at the present position if the transports are not well planned. The proposed approach repetitively and arbitrarily selects as many orders as cranes are available from a static set of orders and arranges these orders to a collision free cluster, calculates possible transition times between all arranged clusters, and calculates the best sequence of the clusters using dynamic programming. Although the approach was only tested for smaller instances and using static data, the authors propose to embed it into a

moving window approach to handle large-scale problems and unknown order arrival times in a decomposed fashion. 1.4. Timed Automata Based Scheduling. A relatively new method to model and solve scheduling problems using a graph-based search is by means of reachability analysis of timed automata (TA). Timed automata were introduced by Alur and Dill55 to model and to analyze real time systems with discrete dynamics. Timed automata are an extension of finite state automata by introducing real valued clocks. The timed automata framework offers a graphical problem representation and model building in a modular fashion, making the approach intuitive and comprehensible also for inexperienced users. Due to the modular nature of the framework, changes in the model formulation can easily be implemented. Many contributions in the area of timed automata-based scheduling have been made in the past decade, starting with the contributions from Fehnker,56 Abdeddaim,57 and Abdeddaim and Maler.58 The extension of timed automata to priced timed automata (Behrmann et al.,59 Larsen et al.,60 Behrmann et al.61), which includes costs for transitions and cost rates for periods of staying in locations, further motivated use of the approach for scheduling problems, as it can be used to formulate more complex objective functions. Cost-optimal reachability analysis for priced TA aims at finding a path from the initial state to a target state with optimal cost. Similar to many other approaches that solve scheduling problems, the TA-based approach also suffers from the problem of the combinatorial explosion, thus demanding efficient pruning procedures to reduce the search space. Panek et al.62 proposed advanced state-space reduction techniques based on the sleep-set method proposed by Godefroid63 to enhance the efficiency of the reachability analysis to solve job shop problems. In ref 64 the authors extended the sleep-set reduction technique for job shop problems to problem classes with more complex constraints. In ref 65 the idea of computing lower bounds for makespan minimization problems for job shop problems by embedding linear programming (LP) into the cost-optimal reachability analysis was introduced to reduce the search space. Subbiah et al.66 and Schoppmeyer et al.67 presented a new, more efficient bounding technique based on the minimum remaining processing time (MRPT), which provides lower bounds to reduce the search tree without much computational effort. The application of the TA-based scheduling approach to handle job shop problems with alternative paths and due dates was presented in ref 68. Panek et al.64 extended the TA-based approach for job shop problems to multiproduct and multipurpose batch scheduling and compared it to MILP techniques for benchmark problems from the process industries. The problem of modeling changeover procedures has been addressed in refs 69 and 70. The problem of modeling various storage policies and operational policies has been addressed ref 71. One of the challenging aspects of scheduling is the uncertainty in the relevant information. Almost every work contributed previously on TA-based scheduling considered solving the problem presuming that all relevant information on the problem is known a-priori and is deterministic. The only contribution in the nondeterministic direction was by Abdeddaiem et al.,72 where the authors extended the TA-based approach to tackle job-shop problems with uncertainties in task durations. 1.5. Scope of the Contribution. In terms of chemical batch scheduling, this contribution falls in the category of reactive scheduling approaches toward scheduling under uncertainties. C



Article

Following the classification of Vieira et al.39 for production scheduling in manufacturing environments, this contribution falls in the category of dynamic scheduling with arrival variability and path variability (infinite set of jobs). The rescheduling strategy is predictive-reactive with an adaptive periodic rescheduling policy. The rescheduling method generates nominal schedules using complete regeneration. The main aim of this contribution is to extend the TA-based approach toward solving dynamic, stochastic scheduling problems. We propose a solution approach based on timed automata and reachability analysis to solve scheduling problems with information that is delivered online, and the scheduler reacts to the online information and delivers better results than the currently used dispatching heuristic. The graphical and modular nature of the modeling framework using TA makes it a suitable candidate for modeling the shuttle management problem with its high degree of flexibility in terms of alternative operation sequences for the pallet removal jobs and the crucial aspect of tracking the shuttle positions at any time, as explained below. By virtue of the modularity of the modeling framework, it is to combine different automata to form the optimization model for the next optimization window in the moving window approach. As pointed out by Gu et al.,73 the dynamic nature of the operational problems in warehouse systems demands procedures that can find a good solution reliably in a fixed amount of time. For the MILP approach it is in general not known if and when the first feasible (integer) solution is found. Hence, finding such first solution in a hard constraint amount of time is not reliably possible using the MILP approach. In contrast, the TA-based approach is able to find a feasible initial solution within fractions of seconds64,74 due to its graph-based nature, in which finding a path from the root node to a target node only requires the evaluation of a few tens of nodes and every such path represents a feasible schedule. In refs 64 and 74 it was shown that the TA-based approach can find better solutions for difficult problems faster compared to a MILP approach. Hence, the TA-based approach is a suitable candidate for the problem at hand and will be evaluated in comparison to the existing dispatching rule, following the research direction emphasized by Vieira et al.39 to compare predictive-reactive scheduling polices with dynamic scheduling policies that consist of dispatching rules. The paper is structured as follows: in section 2 the shuttle management problem in a polymer production plant is introduced. The requirements for a reactive solution approach are highlighted. In section 3 the timed automata (TA) based modeling framework to model a scheduling problem by considering it as a discrete event system with timed behavior is explained along with a simple illustrative example. Following this, in section 4 the TA model of the scheduling problem of the case study is explained in detail. In section 5 the cost-optimal reachability analysis algorithm to solve a scheduling problem, modeled as a timed automaton, is briefly explained. In section 6, the core idea of embedding the TA-based solution into a moving window approach that provides a dynamic solution approach to scheduling problems with uncertainties is explained. In section 7 an extension of the currently used first-come, first-served dispatching rule is introduced. This problem-specific heuristic is considered as the reference solution to which the TA-based solution has to be compared. In section 8 the performance of the two approaches is discussed. Finally section 9 closes the

contribution with conclusions and potential directions for future work. The concept of the moving window approach with TA-based scheduling was published first in ref 75.

2. PALLET REMOVAL IN A HIGH-RISE WAREHOUSE The problem investigated here came up in a polymer production plant located in Gelsenkirchen, Germany. The plant produces 50−60 different types of solid polymers. The products either are stored in large dedicated silos (only one type of polymer per silo) or are packed in 25 kg plastic bags. The bags are stacked on pallets (only one type of polymer per pallet) and stored in a high-rise warehouse (HRW). From both types of storage, the silos and the HRW, the products are distributed to the customer by truck, either silo trucks or pallets trucks. A pallet truck usually carries between 16 and 18 pallets of different products. The part of the plant which is considered here is the pallet removal from the HRW to the trucks. A first model of this system was defined and used for moving window scheduling in ref 75. Based on new information from the real plant, the model was modified as discussed below. The layout of the HRW is shown in Figure 1. The HRW consists of the following parts: a storage area (shown on top of the figure) with a maximum capacity of 24,800 pallets in which the different products are organized in 12 different racks; six pallet retrieval machines (PRM, not shown) which store or retrieve the pallets in or from two racks each; a pallet conveying system that consists of six pallet conveyor lanes (PCL), labeled Lane 1 to Lane 6, which are associated with the six PRMs; a shuttle system that consists of two automated shuttle robots on a single track shuttle lane, labeled Shuttle 1 and Shuttle 2; six shipping output conveyors (SOC), labeled SOC 1 to SOC 6, which are aligned with the PCLs except for Lane 1 and SOC 1; a shipping input conveyor (SIC); a forklift area with manually operated forklifts; and a shipment area (shown at the bottom of the figure) where the trucks which distribute the pallets to the customers are parked during the loading process. The physical pallet removal process works as follows: based on the customers’ demands, the pallets that contain the ordered polymer are retrieved from the storage area by the PRMs. A PRM can carry only one pallet at a time. The PRM drops the pallet after the retrieval on the associated PCL. The PCL automatically conveys the pallet to the end of the lane at the singletrack shuttle lane. A PCL can convey and hold only one pallet at a time. Once the final position on the PCL is reached, one of the shuttle robots is assigned to execute the transport of the pallet to the required SOC and is moved to the corresponding position in front of the PCL and picks up the pallet. The shuttle transports the pallet to the target position in front of the SOC and drops it. Then the pallet is removed from the SOC by a manually operated forklift and is placed on a truck in the shipment area. The pallet removal process is tracked and managed by a control system, called a high-rise warehouse supervisory system (HSS). For each pallet removal, a telegram is created which contains the source location of the pallet in the storage area and the target location of the pallet (one of the SOCs). The telegram is sent to the storage area and to the shuttle management system (SMS) at the same time. In the storage area, the pallet is retrieved by the responsible PRM and dropped on the aligned PCL. At the end of the automatic conveying of the pallet toward the shuttle lane, the pallet passes a light beam (marked as LB1 to LB6 in Figure 1) which signals the pallet arrival to the D



Article

Figure 1. Layout of the high-rise warehouse (HRW).

Figure 2. Positions for the shuttles on the single-track shuttle lane of the high-rise warehouse (HRW).

Transporting a pallet using one of the shuttles is executed by the following steps: (i) the assigned shuttle moves to the source PCL, (ii) the pallet is picked up by the shuttle, (iii) the shuttle transports the pallet to the target SOC, (iv) the pallet is dropped on the SOC. In the case when a shuttle is already in front of the source PCL of the pallet, the movement to the PCL in part (i) is skipped if this shuttle is assigned to execute the removal. If the source PCL of the pallet is aligned with the target SOC of the pallet (e.g., transporting a pallet from PCL 2 to SOC 2), the transport of the pallet by the shuttle in part (iii) is skipped and the pallet is directly dropped after the pickup by the shuttle. The time required by a shuttle to pick up a pallet or to drop a pallet is 20 s. Figure 2 shows the positions on the single-track shuttle lane (marked as P1 to P8) which have to be reached by the shuttles to pick or drop pallets from PCL 1 to PCL 6 and SOC 1 to SOC 6, respectively. The time required by a shuttle to move from any of the positions to its direct neighbor, e.g.

SMS. The exact times taken to retrieve a pallet by a PRM and to convey the pallet on one of the PCLs are not known, since they depend on the pallet position in the rack and on the different mechanical speeds of the PCLs. Hence, the exact pallet arrival times at the shuttle lane are unknown to the SMS until the light beam signal is received. When a light beam signal for a pallet is received, the transportation job is assigned to one of the shuttles and the transport can be executed by the selected shuttle. In total, there exist 36 pallet routes, every combination between the six PCLs and the six SOCs. However, due to practical limitations of the single-track shuttle lane, not all of them can be executed by both shuttles. All pallet routes which start in PCL 6 or end in SOC 6 can only be executed by Shuttle 2, since Shuttle 2 occupies the position of the shuttle lane at which PCL 6 and SOC 6 are located when it is moved to the extreme right-end. A shuttle can only carry one pallet at a time, an exchange of a pallet is not possible, and a shuttle cannot overtake the other since the shuttle lane has only a single track. E



Article

starts with the first four orders being ready to be transported at the end of the PCLs and the shuttles in their initial positions at the extreme-left of the shuttle lane for Shuttle 1 (position P1) and at the extreme-right of the shuttle lane for Shuttle 2 (position P8).

from position P6 to position P5, is 10 s. It should be noted that the durations of the movement and transportation tasks depend on the current position of the shuttles. Any longer movement will add an additional 10 s per position; for example, the movement of a shuttle from position P2 to position P6 has a total duration of 40 s. In the current system, up to 1080 pallets per 16-h work day can be processed (i.e., removed from the warehouse). In general, it is possible to insert pallets into the HRW using the SIC and PCL 1 to PCL 3, but in the current operating policy no pallets are inserted via the SIC during the considered 16-h work day. The current shuttle management system uses a first-come, first-served dispatching rule to assign and execute the pallet removals. The goal of the study is to increase the pallet throughput by at least 33% to 1440 pallets per 16-h work day using an optimal shuttle management that minimizes the average pallet transportation times. The following characteristics make this a challenging problem: • The arrival times of the pallets at the end of the PCLs from where they can be picked up by the shuttles and the pallet routes are not known in advance. • The two shuttles share the same single-track shuttle lane and must not collide with each other. • The shuttles cannot pass or exchange pallets. • Pallet routes which start from PCL 6 or end in SOC 6 can only be handled by Shuttle 2, since the position in the shuttle lane cannot be accessed by Shuttle 1. • Certain decisions on the assignment of the pallet transportation to the shuttles might lead to blocking of the singletrack shuttle lane, e.g. a situation where Shuttle 1 should transport a pallet from PCL 1 to SOC 5 and Shuttle 2 is assigned to transport a pallet from PCL 4 to SOC 1. Then the shuttles block each other if at the same point in time Shuttle 1 picks the pallet from PCL 1 and Shuttle 2 picks the pallet from PCL 4. In this case, additional movements such as one of the shuttles giving way to the other by moving back in the lane have to be considered. In the example either Shuttle 1 should wait until Shuttle 2 has dropped its pallet on SOC 1 and gives way, or Shuttle 2 should move to position P8 after picking up its pallet to give way for Shuttle 1 to drop its pallet on SOC 5. An optimal shuttle management should meet the following requirements: • Include the attributes and characteristics of the HRW system as listed before, e.g. avoid blocking in the shuttle lane. • Have the ability to react to a continuous stream of information about the arrivals and routes of new pallets. • Feasible and good solutions must be returned within short computation times. The durations of short shuttle movements and of picking and dropping of pallets are 10 and 20 s, respectively. The pallet routes occur randomly, and the empirical probability distribution is shown in Table 1. A working day

3. MODELING FRAMEWORK: TIMED AUTOMATA In this section, a brief introduction to timed automata (TA) is presented and the procedure to model a scheduling problem using TA is explained by means of an illustrative example. Timed automata are finite state automata extended by the notion of clocks to model discrete event systems with timed behaviors. The values of the clocks are initialized to zero at the start of the system and at any point in time increase at the rate ċ = 1. The clocks in a TA can be reset to zero but cannot be stopped. After a clock reset action depicted by the label c := 0, the valuation of the clock increases again. The states of a TA are composed of discrete locations, similar to states in finite state automata, and vectors of clock values. Discrete transitions in TA switch from one location to a neighbor location while time transitions wait in a location and increase the values of all clocks by the same value. A detailed formal definition and introduction to syntax and semantics of TA and the modeling procedure can be found in refs 59 and 70. In the illustrative example process, two products A and B are manufactured according to given recipes in a production environment with two units U1 and U2. The recipe to produce one batch of product A consists of two operations: opA1 followed by opA2. The production step opA1 is processed in unit U1 for 3 time units. Following this step, the next operation opA2 is processed in unit U2 for 8 time units to produce the final endproduct A. Similarly, the recipe for one batch of product B consists of two operations: opB1 succeeded by opB2. Operation opB1 is processed in U2 for 9 time units, and operation opB2 is processed in U1 for 3 time units to produce the final endproduct B. The time taken to transfer the materials between the processing units is neglected. Additionally, to highlight the principle of shared variables, a power consumption limit is introduced for the plant. The total amount of available power is limited to 10 units at a time. Hence, a shared variable SP to model the amount of power available is introduced and initialized with a value of 10. For each operation of the two recipes the following power consumption is assumed: opA1 requires 7 units of power, opA2 requires 4 units of power, opB1 requires 5 units of power, and opB2 requires 3 units of power. The goal is to produce one batch of product A and one batch of product B with minimal makespan without violating the power consumption limit. When modeling the example process described above by a network of TA, the following restrictions are taken into account:

Table 1. Pallet Route Probability Distribution Target SOC Source PCL

PCL PCL PCL PCL PCL PCL

1 2 3 4 5 6

SOC 1

SOC 2

SOC 3

SOC 4

SOC 5

SOC 6

5.67% 5.19% 5.04% 0.08% 0.09% 0.10%

6.22% 6.07% 5.75% 0.37% 0.14% 0.18%

3.68% 3.49% 3.87% 1.03% 0.62% 0.68%

1.24% 1.26% 1.39% 4.64% 4.25% 4.01%

0.20% 0.06% 0.14% 4.94% 4.75% 4.91%

0.11% 0.11% 0.21% 6.02% 6.99% 6.52%

F



Article

• Each operation of a recipe can only be in one of these states: waiting to execute in a resource, executing in a resource, or finished. • Each resource can only be in one of these states: idle and not performing an operation or busy executing an operation assigned to it. • Preemption is not allowed; hence, a recipe operation may not be interrupted and thus after starting its execution, the resource occupied by it can only be released after the operation has been completed. • At any point in time a recipe operation cannot be executed in more than one resource. • At any point in time a resource cannot execute more than one recipe operation. In the following paragraphs the modeling of the example process introduced above using TA is described. The process is modeled in a modular fashion where the recipes and the resources are modeled individually as TA and represented as recipe automata and resource automata. The modular TA model of the example process is shown in Figure 3.

Each recipe automaton contains a clock to model the timing of the operations. In the recipe automaton A the clock c1 measures the duration of the operations with respect to recipe A. Every transition that represents starting of an operation contains an action of resetting the clock of the corresponding recipe automaton to 0. Every execute location of an operation (exec op) has an invariant condition, and every transition from an execute location of an operation (φi) has a guard condition. The invariant condition in the location states that the location must be active only for an elapsed duration which is less than or equal to the given duration of the corresponding operation. The guard condition states that the operation should be executed for at least the given processing duration before starting the next operation. The guard conditions and the invariant conditions ensure that the operations are executed for the corresponding durations only. For example the invariant c1 ≤ 3 in the location exec op1A of operation op1A forces the recipe automaton R1 to leave the location exec op1A once the duration of 3 units has expired. The guard c1 ≥ 3 on the finishing transition of the operation op1A ensures that the operation is executed for exactly 3 time units. To model the power consumption of the recipe operations, guards on the shared variable SP on the transitions leading to the execute locations of an operation (αi) and actions on the shared variable SP on the transitions leading to the execute locations and from the execute locations (φi) are introduced. The guards ensure that the required amount of power is currently available for the corresponding recipe operation started by the transition; for example, the guard SP ≥ 7 is introduced for the transition labeled α1 starting recipe operation opA1 which required 7 units of power. The actions on the transition to and from the execute locations ensure the consumption and release of the required power of the recipe operations; for example, the action SP := SP − 7 on the transition labeled α1 consumes the required 7 units of power when the operation is started and the action SP := SP + 7 on the transition labeled φ1 releases the required 7 units of power when the operation is finished. The automaton B represents the recipe automaton for recipe B and is modeled using the same procedure as explained above. 3.2. Resource Automata. For each processing unit a separate resource automaton is created. Each resource automaton consists of a single idle location, which represents that the resource is not occupied, and a busy location, which represents that the resource is executing a recipe operation. For each operation that can be performed in a resource, a transition from the idle location to the busy location and a transition from the busy location to the idle location exists. The transition from idle to busy represents the allocation of the resource in order to start executing an operation, and the transition from busy to idle represents the release of the resource when the corresponding operation has been finished. The transitions on the resources (allocation and release) are synchronized with the corresponding transitions of the recipe automata. For the example considered, the idle location of the resource automaton U1 is represented by idleU1, and the busy location is represented by busyU1. The allocation of the resource U1 when starting the operation op1A of recipe A is modeled by synchronizing the transition from the location wait op1A to the location exec op1A in automaton A with the transition from idleU1 to busyU1 in automaton U1 with the synchronization label α1. Similarly, the release of the resource U1 after finishing the operation op1A is modeled by synchronizing the transition from the location exec op1A to wait op2A in automaton A with the

Figure 3. TA model of the example process.

3.1. Recipe Automata. For each of the recipe orders, a recipe automaton is created. Automata A and B represent the recipe automata of recipes A and B. Each operation of a recipe is represented by two locations, wait, where the operation is waiting to be executed in the corresponding resource, and execute, where the corresponding resource is executing the operation. The wait and the execute locations of the operation op1A are labeled as wait op1A and exec op1A. Since the state which represents that operation op1A has finished its execution is equivalent to the state that operation op2A is waiting to execute, the former can be omitted. Similarly, the state which represents that operation op2A has finished execution is equivalent to the termination of the recipe A, so the former is omitted and a location called fA is defined to indicate the completion of the recipe A. Transitions that represent starting of an operation i and finishing of an operation i are represented with labels αi and φi. Starting the execution of the operation op1A in the resource U1 is represented by a transition labeled α1 and finishing the execution of the operation is represented by a transition labeled φ1. The precedence constraint between the operations op1A and op2A in recipe A is modeled by the transition from the location exec op1A to wait op2A with label φ1. This ensures that operation op2A can start only after executing operation op1A. G



Article

transition from the location busyU1 to idleU1 in the automaton U1 with the synchronization label φ1. Using the same conceptual ideas explained above for the recipes and the resources of the simple example, the TA model of the shuttle system is set up.

highlighted in orange, represents the case when the removal process is executed by Shuttle 2. It should be noted that the state f inished execution of an operation is equivalent to the state waiting to be executed of the succeeding operation. Hence, the former is omitted whenever possible. The abstract job structure starts in the state wait to start removal, which represents that the pallet has passed the laser beam in the source PCL and is ready to be picked up by one of the shuttles. Depending on the pallet route, either the removal can be executed by both the shuttles or it can only be executed by Shuttle 2 (if it starts in PCL 6 or ends in SOC 6). In the latter case only the states of the lower part of the figure can be reached. Depending on the shuttle chosen and its current position in the lane, either the movement of the shuttle to the pickup position of the source PCL is executed, or the pickup of the pallet is directly executed if the shuttle is already in the right position. The execution of the movements of Shuttle 1 and Shuttle 2 to the pickup position is represented by the states execute move shuttle 1 to pickup lane and execute move shuttle 2 to pickup lane, respectively. The state when the movement is finished is equivalent to the state waiting to start the pickup of the pallet and therefore represented by the single state wait to pickup. Since any waiting in this state would not lead to a better objective with respect to time, e.g. minimization of the makespan or minimization of the lead times (here: the time between the pallet arrival at the end of a PCL and the dropping of the pallet on a SOC), waiting is not permitted in this state; hence, it is marked with a zero wait annotation. The execution of the pickup operation by a shuttle is represented by the state execute pickup. After the pickup operation, either the shuttle has to transport the pallet to the required position at the target SOC, or the pallet can immediately be dropped to the SOC if the source PCL and the target SOC are aligned. In the former case, the shuttle might have to wait to execute the transport of the pallet to the target SOC if the shuttle lane is blocked by the other shuttle, this is represented in the states wait to move shuttle 1 to drop SOC for Shuttle 1 and wait to move shuttle 2 to drop SOC for Shuttle 2. Once the shuttle lane is free, the transport operation can be executed, which is represented in the states execute move shuttle 1 to drop SOC and execute move shuttle 2 to drop SOC for the individual shuttles. The state that represents the completion of the transport operation is the same state as waiting to start the dropping of the pallet to the SOC. Hence, the state wait to drop represents both the above-mentioned states. Since any waiting

4. TA MODEL OF THE SHUTTLE SYSTEM The system is modeled as a flexible job shop since for different pallets different removal routes exist (job shop) and for most of the routes any of the two shuttles can be selected and assigned (alternative routes). For each of the 36 pallet routes a separate job automaton (or recipe automaton) is created following the general structure of the pallet removal process. The shuttles are modeled as resources (or machines); hence, for each shuttle a separate resource automaton is created. Additionally, five shared variables SO1, SO2, SP1, SP2, and SD are introduced to represent if a shuttle is currently occupied by a job (SO1, SO2), to track the current positions of the shuttles (SP1, SP2), and to track the current distance between the two shuttles (SD). Table 2 shows a summary of the shared variables with their initial values and the corresponding descriptions. For modeling the pallet routes as job automata, each basic operation of the pallet removal process can be represented by three different states, namely waiting to be executed, executing, and f inished execution. Figure 4 shows the abstract job structure in terms of states of the basic operations. The upper part of the job structure, highlighted in green, represents the case when the removal process is executed by Shuttle 1, and the lower part, Table 2. Summary of the Shared Variables of the TA-Model of the Shuttle System Shared variable

Value range

Initial value

SO1

{0, 1}

1

SO2

{0, 1}

1

SP1

{0, 1, 2, 3, 4, 5, 6, 7}

1

SP2

{0, 2, 3, 4, 5, 6, 7, 8}

8

SD

{0, 1, 2, 3, 4, 5, 6}

6

Description 1 − Shuttle 1 is idle. 0 − Shuttle 1 is occupied. 1 − Shuttle 2 is idle. 0 − Shuttle 2 is occupied. 1−7 − The current position of Shuttle 1. 0 − Shuttle 1 is currently moving. 2−8 − The current position of Shuttle 2. 0 − Shuttle 2 is currently moving. 1−6 − The distance between the shuttles. 0 − Both shuttles are next to each other.

Figure 4. Abstract recipe structure for the pallet removal process. H



Article

Figure 5. Job automaton for a pallet removal from PCL 6 (F) to SOC 5 (E).

and drop locations (PCL 1 and SOC 1 with A, PCL 2 and SOC 2 with B, PCL 3 and SOC 3 with C, PCL 4 and SOC 4 with D, PCL 5 and SOC 5 with E, and PCL 6 and SOC 6 with F); that is, a job BC represents the pallet route from PCL 2 (B) to SOC 3 (C). The job automaton FE for the pallet route from PCL 6 (F) to SOC 5 (E), which can only be executed by Shuttle 2, is shown in Figure 5. For the job automaton FE, a clock CFE is introduced to model the duration of the job. The initial location wait move1FE represents that the operation of moving Shuttle 2 to the pickup position F is waiting to be executed. Since the duration of the movement operation depends on the current position of Shuttle 2, a discrete location for each accessible position in which it can be is introduced. The locations exec move1_1FE until exec move1_7FE represent the execution of the movement operation from the current position of Shuttle 2 (P2−P8) to the source PCL (F); for example, location exec move1_1FE represents that Shuttle 2 is in position P2 and is required to move to position P8 (F) (this movement operation has a duration of 60 s). Each of the movements is considered as a “safe” movement, meaning there is no possibility for collision, since Shuttle 2 is moving to the right and hence it cannot collide with Shuttle 1. In other words, the lane is always unoccupied for a movement of shuttle 2 in this direction. The special case in which Shuttle 2 does not have to move to the source PCL, since it is already located in front of the PCL, is represented in the location exec move1_7FE. Hence, the duration of this movement is 0 s. The guards (SO2 = 1) on

in this state would not lead to a better objective value with respect to time, e.g. minimization of the makespan or minimization of the lead times, waiting is not permitted in this state and it is marked with a zero wait annotation. The execution of dropping the pallet is represented by the state execute drop. It should be noted that the state execute drop can directly be reached from the state execute pickup if the source PCL and the target SOC are aligned, since any waiting between both operations would not lead to a better objective with respect to time. After finishing the dropping of the pallet, the complete removal for this pallet is finished and the assigned shuttle is released, which is represented in the state removal finished. In general, the jobs of the case study have a high degree of flexibility in terms of alternative operation sequences. One part of the flexibility is introduced by the possibility to choose among the two shuttles for the execution of the pallet removal, and additional flexibilities are introduced in the selection of the appropriate movements for the allocated shuttles. The alternative paths of the shuttle movements are modeled following the main idea of modeling batch scheduling problems with sequence-dependent changeovers using timed automata by additional tasks and shared variables, reported by Subbiah et al.70 and applied to a multiproduct pipeless batch plant70 and a multiproduct pipeless batch plant with mobile robots (Schoppmeyer et al.76). For simplicity reasons, the jobs are abbreviated in the automata models with capital letters A−F based on their pick-up I



Article

the shared variables of the transitions from the location wait move1FE to the locations exec move1_1···7FE ensure that Shuttle 2 is unoccupied before the movement is started (SO2 = 1). In addition, the guards (SP2 = 2···8) ensure that only the location that corresponds to the current position of the shuttle is active. The actions on the transitions update the shared variables with values that represent that Shuttle 2 is occupied for the pallet removal (SO2 := 0) and indicate that the position of Shuttle 2 is currently changing due to a movement of the shuttle (SP2 := 0). Additionally, the clock CFE is reset on the transitions (CFE := 0). Invariants in the exec move1_1···7FE locations (e.g., CFE ≤ 60 in the location exec move1_1FE) and guards on the transitions leaving these locations (e.g., CFE ≥ 60 on the transition leaving the location exec move1_1FE) ensure that the movement operations are executed for the defined durations only. By the transitions leaving the locations exec move1_1···7FE, the location wait pickFE is reached that represents that the shuttle has reached the position in front of the source PCL and is ready to pick up the pallet. Since any waiting in this state would not lead to a better objective with respect to time, the clock CFE is reset on the transitions leading to this location and an invariant CFE ≤ 0 is introduced in the location to restrict the waiting time to zero. The location exec pickFE represents that the pallet is picked up by the shuttle. The clock reset on the incoming transition, the invariant in the location, and the guard on the outgoing transition ensure that the operation is executed for the defined duration only. The location wait move2FE represents that Shuttle 2 has finished picking up the pallet and is waiting to start the transport of the pallet to the target SOC. Since for the transport of the pallet from the source PCL to the target SOC a movement to the left of Shuttle 2 is required, the occupancy of the single-track shuttle lane has to be checked. Either the shuttle lane is blocked by Shuttle 1 and a simultaneous movement of both shuttles has to be executed to give way for the movement of Shuttle 2 or the shuttle lane is free, meaning that Shuttle 1 has a current distance of more than one free position to Shuttle 2 and thus Shuttle 2 can execute the transport. The simultaneous movement of both shuttles is represented in location exec move2_1FE. The transport of Shuttle 2 is represented in location exec move2_2FE. The guards on the shared variables on the transition from location wait move2FE to exec move2_1FE ensure that the shuttle lane is blocked by Shuttle 1 (SP1 = 7) and that Shuttle 1 is idle (SO1 = 1). If the conditions are met, a simultaneous move can be executed. The actions on the same transition block Shuttle 1 (SO1 := 0) and indicate the movements by updating the shared variables (SP1 := 0 and SP2 := 0). After the movement, Shuttle 1 is released (SO1 := 1) and the positions are updated by the actions on the transition (SP1 := 6 and SP2 := 7) leaving the location exec move2_1FE. The guards on the shared variables on the transition from location wait move2FE to exec move2_2FE ensure that the shuttle lane is free by checking the distances between the shuttles (SD ≥ 1). The actions on the same transition indicate the movement of Shuttle 2 (SP2 := 0) and update the distance between the shuttle according to the movement (SD := SD − 1). The new position of Shuttle 2 is set (SP2 := 7) when the movement is finished on the transition leaving the location exec move2_2FE. The location wait dropFE represents that Shuttle 2 has reached the position in front of the target SOC and is ready to drop the pallet. Since any waiting in this state would not lead to a better objective with respect to time, the clock CFE is reset on both incoming transitions of this location and an invariant CFE ≤ 0 is introduced in the location. The location exec dropFE represents

that the dropping of the pallet at the SOC is executed. The location f inishFE represents that the removal of the pallet is finished. Hence, the action on the transition from exec dropFE to f inishFE releases Shuttle 2 by updating the shared variable (SO2 := 1). Figure 6 shows the job automaton CC for the pallet route from PCL 3 (C) to SOC 3 (C), which can be executed by both

Figure 6. Job automaton for a pallet removal from PCL 3 (C) to the aligned SOC 3 (C). J



Article

shuttles. For the job automaton CC, a clock CCC is introduced to model the timing behavior of the job. For the removal process, it is required that any of the two shuttles moves to position P5 (C), picks up the pallet from the PCL, and immediately drops the pallet to the aligned SOC. The execution of the movement of Shuttle 1 to the source PCL is represented by the locations exec move1_1CC to exec move1_11CC. The execution of the movement of Shuttle 2 to the source PCL is represented by the locations exec move1_12CC to exec move1_21CC. The actual movement of a shuttle and its duration depends on the current positions of both shuttles. Either the shuttle lane is free and the shuttle can directly move to the source PCL, or the shuttle lane is blocked and a simultaneous movement of both shuttles is required to free the shuttle lane. The guards on the transitions from the initial location wait move1CC to the locations representing the movement executions exec move1_1CC to exec move1_21CC model the different combinations of the shuttle positions and the shuttle occupancy. The actions on the shared variables of these transitions and of the transitions leaving the movement execution locations ensure that the positions and the occupancies are updated correctly. The invariants in the movement execution locations and the guards on the clock CCC of the outgoing transitions of these locations ensure that the corresponding movements are executed for the required durations only. After the movement execution, when position P5 (C) is reached, the pallet is waiting to be picked up by the shuttle and be immediately dropped, which is represented in the locations wait pickdrop_1CC and wait pickdrop_2CC, depending on which shuttle is selected for the removal, Shuttle 1 or Shuttle 2, respectively. Since any delay in one of the locations would not lead to a better objective value with respect to time, the active time of the locations is restricted to zero. The execution of the pick-up and immediate dropping is represented by the locations exec pickdrop_1CC and exec pickdrop_2CC. After dropping the pallet on the target SOC, the pallet removal is finished, which is represented in location f inishCC. The actions on the shared variables on the incoming transitions of the f inishCC location release the shuttle, which was used for the removal (SO1 := 1 or SO2 := 1). Similarly to the job automata FE and CC, all other 34 pallet routes are modeled as job automata, and for each of the job automata, a clock is introduced. For each of the shuttles, a separate resource automaton is introduced with two locations each: an initial idle location and a busy location. Transitions between the idle and the busy locations are introduced with labels and synchronized with the corresponding transitions in the job automata of the pallet removals to ensure that the shuttle execute at most one operations at a time. The composition of the 36 job automata with their clocks, the two resource automata, and the five shared variables defines the complete model of the scheduling problem. The generation of the job automata was automated based upon the abstract recipe structure and following the idea of modeling the shuttle movements as sequence-dependent changeovers, as presented in detail above for two exemplary timed automata. This automatic procedure works as follows: first, it is checked whether the route can be executed by both shuttles or if it can only be executed by shuttle 2. If it can be executed by both shuttles, an alternative branch for each shuttle is created in the TA model for the pallet route. If it can only be executed by shuttle 2, only a single branch is created. In each of the branches to move the selected shuttle to the right position

for the pickup of the pallet, all combinations of possible shuttle positions and movements are enumerated as alternative paths. After the movement, the pickup task is inserted. After the pickup, and if required by the pallet route, again all combinations of possible shuttle positions and movements are enumerated as alternative paths. Finally, the drop task is added. The effort spent to develop the systematic modeling approach for the pallet removal case study and to fully automate the model building was moderate. The first version of the models (as reported by Subbiah et al.75) was created within the scope of a Masters thesis by an inexperienced (in terms of TA-based scheduling) student.77

5. SOLUTION APPROACH: REACHABILITY ANALYSIS The sets of job automata and resource automata are combined by the technique of parallel composition, see ref 55, to form a composed global automaton that represents the complete scheduling problem as a directed graph. The directed graph of the problem has an initial node where no operation has started and all resources are idle, and a target node where all operations are finished and all resources are idle again. Every path from the initial node to the target node represents a feasible schedule. The shortest path according to a given cost criterion among all paths from the initial node to the target node represents the best or optimal schedule. A cost-optimal reachability algorithm for TA models which computes the shortest path was presented by Panek et al.62,68 The algorithm is a brute-force graph search algorithm that iteratively evaluates the nodes of the directed graph including backtracking. Different graph search techniques, e.g. depth-first search, are used to guide the exploration of the graph into promising regions; see Panek et al.64 To enhance the solution performance, a branch-and-bound scheme (Panek et al.,64 Schoppmeyer et al.67) and other state space reduction techniques (Panek et al.64) are embedded into the algorithm. The exploration scheme is stopped when the whole search space has been explored and the optimality of the best found solution is proven, or when a specified termination criterion is reached, e.g. a computation time limit, a limit for the number of explored nodes, or a limit for the number of solutions found. Cost-optimal reachability algorithm for TA models; see Panek et al., 2006

K



Article

Figure 7. Structure of the dynamic scheduling solution which embeds the TA-based approach into a moving window approach (MWA).

current state of the plant, and a control horizon that defines which part, e.g. corresponding to 75% of the makespan, of an optimized schedule will be implemented on the system before the rescheduling is triggered. According to the general principle of a moving window algorithm shown in Figure 8, the proposed dynamic scheduling solution operates iteratively by first calculating an optimized schedule for a predefined set of job operations, the prediction set, of the initially available orders, e.g. all job operations of the first four orders. Only the first part of the optimized schedule, the control horizon, e.g. defined as 75% of the makespan of the calculated schedule or by a point in time calculated based on the utilization of the units in the schedule, is fixed and implemented in the system. The remainder of the schedule is kept unfixed and can be changed and optimized according to newly revealed information in the next iteration. During the calculation and the execution of the control horizon, all newly revealed information, i.e. the new orders, is collected and stored. In each iteration, the MWA operates as follows: When the rescheduling is triggered, the current state of the plant is provided, including the positions of the shuttles, the distance between the shuttles, and the current job allocation of the shuttles, the current state of the schedule execution, the collected information about newly arrived pallet removal jobs, and the prediction set. Since the scheduler requires a certain amount of time to calculate an optimized schedule, called the response time (tres), the rescheduling is not triggered at the end of the control horizon but tres time units before this time. The prediction set includes a subset of job operations for the next optimization from the set of all available open orders and the remaining partially finished orders. Based on the selected subset of job operations, the model database, and the response time, the model updater updates the TA models for the TA-based scheduler. In order to take the response time of the algorithm into account, the TA models are updated according to a predicted state of the plant according to the existing schedule and not for the currently reported state of the plant when starting the rescheduling execution; for example, for the rescheduling for Window 2 (w2) in Figure 8 at time point “Start rescheduling W2”, the state of the plant at the end of “Control horizon (w1)”, in which the job operations 3B and 4C are finished and job operation 4D is started, defines the predicted state of the model updater and is given to the TA-based scheduler. The scheduler calculates a new optimized schedule for the next window based on the updated model and on some parameters, e.g. computation time limit. From the resulting optimized

The technique of parallel composition and the cost-optimal reachability algorithm, including different guiding techniques for search space exploration and upper and lower bound cuts, and different state space reduction techniques are implemented in the TA-based scheduling tool TAOpt (Timed Automata optimizer), which has been developed in the Process Dynamics and Operations Group at TU Dortmund in the programming language C#78 using the DOT.NET framework.79

6. DYNAMIC SOLUTION APPROACH: MOVING WINDOW APPROACH The unknown arrival times and the a priori unknown routes of the pallets require a dynamic optimization technique to handle the arrival of new information in the solution approach. A dynamic scheduling solution which embeds the TA-based approach to modeling and solving scheduling problems into a moving window approach (MWA) and which is able to tackle the online information on the arrival times and the routes of the pallets has been developed. The general structure of the reactive solution approach is shown in Figure 7. On the right-hand-side, the control system of the pallet removal system (or a simulation) is shown, which consists of the shuttle management that executes the calculated schedules by sending the control inputs to the actuators of the shuttle robots and receiving the sensor feedback. An order generator is introduced in the simulation which generates orders randomly in terms of arrival times and pallet routes for the scheduling problem according to the probability distribution in Table 1. From the data that was recorded at the real plant, it can be seen that after a pallet is picked up from a lane, the retrieval of a new pallet by the corresponding PRM and the conveying of the pallet until it reaches the correct position to be picked up by the shuttle system requires 25 s on average. A new pallet is not retrieved before the corresponding lane is free. Hence, the arrival time of a new order (pallet) at a lane after the previous pallet was picked from the same lane is uniformly distributed in the interval 15−35 seconds. On the left-hand-side, the moving window approach with its components and the embedded TA-based scheduling is shown. The approach consists of a prediction set which includes all known open orders or a subset of these from a database storing all open orders and newly arrived orders, a model updater which manages the TA models for the TA-based scheduler, a model database including all available resource automata and job automata of the 36 pallet routes, the TA-based scheduler for calculating optimized production schedules based on the L



Article

Figure 8. Schematic diagram of the moving window approach.

7. REFERENCE SOLUTION: ENHANCED FIRST-COME, FIRST-SERVED DISPATCHING RULE In order to evaluate the effectiveness and the efficiency of the proposed dynamic scheduling solution, an enhanced version of the original first-come, first-served dispatching rule (in the following termed as EDRenhanced dispatching rule) is taken as the reference point. The original dispatching rule which is currently used in the real plant operation is able to dispatch 1080 pallet removals per 16-h work day but is not capable of realizing the target throughput of 1440 pallet removals within 16 h. The unbalanced load distribution between the shuttles was identified as the main limitation of the original dispatching rule. The rule assigns the new orders to the shuttles based on the distances between the shuttle positions and the lane in which the pallet arrives at the point in time of the arrival. This simple assignment of the pallet removals to the shuttles causes the unbalanced load distribution; for example, in a situation when Shuttle 1 is in position P5 and currently executing a transport to SOC 1 and Shuttle 2 is in position P8 and idle, all new arriving orders in lanes 1−4 are assigned to Shuttle 1 independent of the target SOC and the current load balance between the shuttles. In order to increase the throughput of the original dispatching rule, an enhanced version of the rule that was developed by De La Fuente Valdez is needed.77 The enhanced rule dispatches new arriving orders not only based on the current state of the plant, e.g. the shuttle positions, at the point in time of the arrival, but also according to the load balance between the shuttles specified under the following conditions, as shown in Algorithm 2: • The order o that arrives first in any PCL is scheduled first by selecting it from the set of available unscheduled orders unassigned_orders using the function selectNextOrder(). • If an order o starts in PCL 6 (o.sourcePCL = PCL6) or ends in SOC 6 (o.targetSOC = SOC6), it can only be handled by Shuttle 2. Hence, it is assigned to Shuttle 2 using the function assign(Shuttle2, o).

schedule the control horizon is determined and a new schedule is defined for execution over this period and sent to the plant or to the simulation. Following the principle of the moving window approach, the online revealed information on newly arrived orders is included in the optimization in upcoming windows. The windows of the optimization are overlapping to a certain extent defined by the length of the resulting schedule and the control horizon for that schedule. It is possible that an order which is scheduled for execution at the end of a schedule and not part of the control horizon of the same schedule may again be scheduled for execution at the end of the schedule and once again not part of the control horizon of the new schedule. Such a situation, which we call the creation of a starving order, is undesired since it would mean that a truck might be waiting long for the starving order to be finished. Therefore, the proposed MWA is augmented with a mechanism to check the calculated schedules for such starving orders and to modify the control horizon such that either the original schedule is executed if all operations of all starving orders are already included in it or the control horizon is extended to include all job operations of all starving orders and executed. For the shuttle management problem, starving orders were defined as orders which are completely or partially present in the list of open orders or in the prediction set for more than two successive time windows. The proposed MWA as well as the simulator are implemented in the programming language C# using the DOT.NET framework. The control commands for the shuttle management are passed from the MWA to the simulator in the form of a task list. The current state of the execution of a schedule and of the plant are fed back to the MWA in the same format extended with flags for each operation in the scheduling listing, indicating whether the operation was already executed, is partially executed, or was not started. Internally, the MWA uses the timed automata model of the scheduling problem as the representation of the data. This timed automata model is updated by the model updater according to the current state reported from the simulation to the MWA. M



Article

• If an order o can be executed by both shuttles, the distances of the shuttles based on the current positions of the shuttles at the point in time when the order arrives to the source PCL are checked by using the function dist(). If both shuttles are the same distance away from the PCL, then it is assigned to a randomly selected shuttle using the function assign_random(o). • Otherwise, it is assigned to the nearest shuttle. • In order to balance the load of assigned orders per shuttle and to increase the overall shuttle utilization, for each newly assigned order it is checked whether the queue of the other shuttle is shorter, using the remainingQueueDuration() function, which computes the makespan of the orders in the queue for the given shuttle, and a reassignment of the new order to the other shuttle would lead to a shorter overall makespan using the duration() function, which calculates the required duration for executing the order at the end of the queue of a given shuttle. If both conditions are fulfilled, the order is reassigned to the other shuttle. • If a shuttle is executing an order of highest priority, or in other words an order which arrived to the PCL before the order handle by the other shuttle, the other shuttle has to give way in the single-track shuttle lane, independent of the occupancy of the other shuttle (idle, executing an order). Hence, additional movement operations to give way for the other shuttle are included in the queues of both shuttles when assigning new orders using the function assign(). Algorithm 2: Modified first-come, first-served dispatching rule.

The test series consists of a set of problems generated using an instance generator. The number of jobs (i.e., pallets to be transported) is varied from 100 to 2500 (i.e., 100, 250, 500, 1000, 1440, 1500, 2000, and 2500). For explanation purposes, the test instances with the numbers of jobs of 100, 250, 500, and 1000 pallets are considered as Set A and the rest of the test instances are considered as Set B. The instance generator generates job orders such that they follow the pallet route probability distribution as shown in Table 1. For each of the problem configurations, 25 different instances were created using different seed values. Hence, in total 200 (8 × 25) problem instances were tested for each of the approaches. The choice of the problem configurations was made such that a fair comparison with the reference solution using the enhanced dispatching rule (EDR) can be performed and to be as close as possible to the actual situation in the production plant. It should be noted that, due to the online revealed information on the arrival times and the pallet routes, it is in principle not possible to test the different approaches on the same instance, since the generation of new orders depends on the currently free PCLs and the availability of these again depends on the calculated schedules. A comparative study between the EDR and the reactive solution approach using TA models (MWA TAOpt) is presented below. For all tests, the prototypic scheduling tool Timed Automata Optimizer (TAOpt) developed in the Process Dynamics and Operations Group (DYN) is used as the TA-based scheduler in the proposed moving window approach. The TA-based scheduling tool TAOpt provides the functionality to choose the search technique from the set of available search techniques and the reduction technique to prune the solution space from the set of available reduction techniques to explore the reachability graph. For a detailed explanation of the search techniques and reduction techniques, we refer to Subbiah et al.70 In all the tests, a combination of depth-first and best-first search was used. In the case when more than one node with the same depth and cost exist, then one of them is chosen randomly. The state-space reduction techniques, safe sleep-set method, and safe non-laziness scheme as explained by Panek et al.62,68 as well as the lower bounds presented by Schoppmeyer et al.67 were used. The prediction set in each scheduling window is considered to be 100% of the available orders (in maximum, six newly arrived orders waiting in the PCLs and two currently executing orders, one assigned to each of the shuttles), and the control horizon is defined based on the utilization of the shuttles in the best schedule found (the latest point in time of the schedule at which one of the shuttles is idle again; see Figure 8). The reachability analysis in each of the scheduling windows is terminated if one of the following conditions becomes valid: (a) the waiting list is empty or (b) the computation time has reached the maximum limit of 7 CPU seconds or (c) the number of nodes explored has reached the maximum limit of 100,000. The control horizon that determines which part of a calculated schedule will be fixed and implemented on the system is one of the crucial parameters of the moving window approach. In a preliminary study, different control horizons, either makespan-based or utilization-based, were evaluated without the additional mechanism to prevent orders from starving. For the makespan-based control horizon, different settings with shorter (10−30%) or longer (80−90%) horizons were evaluated. For the utilization-based control horizon, a 100% utilization setting was chosen which determines the control

The described enhanced dispatching rule is executed each time a new order arrives at the end of a PCL. The improvement introduced here over to the original dispatching rule is the load balancing between the shuttles using the remainingQueueDuration() function. The described enhanced dispatching rule is also implemented in the programming language C# using the DOT.NET framework and can be used with the simulator described in the previous section. In the real plant, the original dispatching rule is implemented on a programmable logic controller.

8. NUMERICAL RESULTS In this section, to demonstrate the performance of the proposed dynamic scheduling solution using TA models, numerical tests for a series of self-generated problem instances based on the data of the real plant are presented. The computational equipment used for all the tests performed was an Intel i7 vPro machine with speed of 4 × 3.4 GHz and 8 GB of main memory, operating with a Windows 7 platform. N



Article

horizon as the first part of a schedule until the latest point in time at which the utilization drops from the desired 100% to a lower value. In the case of the pallet removal process, the selected point in time is equal to the point in time at which at least one of the shuttles becomes and remains idle for the rest of the schedule. Figure 9 presents the comparison results on the

25 runs each. The results show that for all instances MWA TAOpt provides schedules where the average and the maximum idle times of each of the shuttles are smaller in comparison to the average and maximum idle times of the shuttles in the schedules found by the EDR. The average idle times of Shuttle 1 and Shuttle 2 are reduced by up to 95% and up to 92%. The reduction in the idle times can be explained by the fact that in the case of the EDR the exploration of all possible allocations of the jobs to the shuttles is not explored whereas in the case of MWA TAOpt the possibility of backtracking the reachability tree when a feasible solution is found in order to explore other solutions is included. Figure 13 presents the comparison results on the average execution, waiting, and lead times of the jobs (in seconds) in the schedules calculated by the EDR and MWA TAOpt for all test instances as the averages of 25 runs each. The results show that for all instances MWA TAOpt provides schedules where the average execution, waiting, and lead times of the jobs are less in comparison to the average execution, waiting, and lead times of the jobs in the schedules found by the EDR. The average execution times of the jobs are reduced by up to 15%. The average waiting times of the jobs are reduced by up to 25%. The average lead times of the jobs are reduced by up to 23%. The reduction in the execution, waiting, and lead times can be explained by the fact that in the case of the EDR the allocations of the jobs to the shuttles are based only on the current positions of the shuttles and the queue lengths whereas in the case of MWA TAOpt all possible assignments of all jobs for the next window are considered and the best solutions in terms of the makespan are executed. Additionally, a comparison of the results on the maximum execution, waiting, and lead times of the jobs (not depicted) shows that the schedules of MWA TAOpt do not include execution, waiting, or lead time outliers whereas there are execution, waiting, or lead time outliers in the schedules computed by the EDR, proving the efficiency of the proposed approach to avoid starving orders in the moving window approach. Figure 14 presents the results on the average load balance of the two shuttles (in percentages of the total job number) in the schedules calculated by the EDR and MWA TAOpt. The results show that the load between the shuttles is not equally balanced in the case of the EDR, although the original firstcome, first-served dispatching rule was improved by the remainingQueueDuration() function to balance the workload between the shuttles. In the case of MWA TAOpt, the load is more equally balanced, since paths representing the allocation of a job to either of the shuttles can be explored within the reachability tree search and the best solution in terms of the objective function values is chosen. In addition to the presented comparative study of the best schedules found by the EDR and MWA TAOpt for the set of instances, a detailed comparison of the results for the reference target instances of 1440 pallets per 16-h work day was made. The results show that MWA TAOpt provides schedules which are in all cases shorter than the 16 h work day limit, whereas the schedules found by the EDR exceed this limit in all cases. In contrast to MWA TAOpt, which provides schedules for the reference target with a spare capacity of 3.3 to 3.7 h of the 16 h limit, the EDR is not able to meet the optimization target of removing 1440 pallets within a 16 h work day. Figure 15 shows the results of the maximum number of pallet removals per 16 h work day (based on the average time per pallet of the test instances) for the old reference, the target reference, and the

Figure 9. Comparison of the lead time distribution of different control horizons for the test instance of 1440 pallet removals.

lead time distribution of the schedules calculated by the MWA TAOpt using different control horizons for the test instances of 1440 pallet removals as averages of 25 runs each. The preliminary results showed that shorter control horizons in the range of 10−30% of the makespan of a window, in other words more frequent rescheduling, lead to small gains with respect to the overall makespan which are traded off by longer maximum lead times (i.e., more starving orders). Longer control horizons (80−90% of the makespan of a window or the utilization-based control horizon) lead to a reduction of the maximum lead times by up to 43% and the utilization-based choice of the control horizon gives the smallest loss relative to the optimum in terms of the overall makespan. For the shuttle management problem, starving orders which can lead to starving trucks in the shipment area are a crucial aspect and must be avoided. Hence, the utilization-based control horizon setting is chosen and combined with the additional mechanism to prevent orders from starving in more than two successive windows. Tables 3 and 4 show the results of the comparison between the EDR and the proposed reactive solution approach using TA models (MWA TAOpt) on the makespan (Cmax), the average and maximum idle times of the shuttles, and the average execution, waiting, and lead times (here, the time between the pallet arrival at the end of a PCL and the dropping of the pallet on a SOC) of the pallet removal jobs for the different sets of problem instances Set A and Set B. Figure 10 presents the comparison results on the makespan values (in hours) of the schedules calculated by the EDR and MWA TAOpt for all test instances as averages of 25 runs each. The comparison results show that for all the instances tested MWA TAOpt provides better schedules in terms of makespan values. The average values of the makespan for schedules found by MWA TAOpt show that compared to the results of the EDR an average reduction of the objective values of 30% is reached. Figures 11 and 12 present the comparison results on the average and the maximum idle times between two jobs of the two shuttles (in seconds) in the schedules calculated by the EDR and MWA TAOpt for all test instances as averages of O



Article

Table 3. Numerical Results for the Test Instances of Set A (Average of 25 Runs per Instance) in Seconds Instance set

100

250

500

1000

Solution method

EDR

MWA TAOpt

EDR

MWA TAOpt

EDR

MWA TAOpt

EDR

MWA TAOpt

Avg makespan Avg idle time Shuttle 1 Max. idle time Shuttle 2 Avg idle time Shuttle 2 Max. idle time Shuttle 2 Avg execution time Avg waiting time Avg lead time

4,525.80 24.20 169.00 11.15 118.80 72.84 196.72 269.56

3,322.00 2.68 63.96 2.77 78.04 63.00 138.92 201.92

11,616.60 25.38 220.80 13.81 142.40 73.60 208.68 282.28

8,033.96 1.66 67.96 1.60 69.04 62.40 152.04 214.56

22,779.00 24.51 239.20 12.27 152.60 73.16 209.84 283.04

15,827.76 1.08 77.68 0.91 62.36 62.20 157.88 220.20

45,515.20 24.56 235.20 12.86 170.00 72.80 212.32 284.96

31,412.84 0.81 83.08 0.76 53.84 62.00 162.64 224.64

Table 4. Numerical Results for the Test Instances of Set B (Average of 25 Runs per Instance) in Seconds Instance set

1440

1500

2000

2500

Solution method

EDR

MWA TAOpt

EDR

MWA TAOpt

EDR

MWA TAOpt

EDR

MWA TAOpt

Avg makespan Avg idle time Shuttle 1 Max. idle time Shuttle 1 Avg idle time Shuttle 2 Max. idle time Shuttle 2 Avg execution time Avg waiting time Avg lead time

65,658.40 24.63 291.20 12.84 185.20 72.92 214.00 287.00

45,089.68 0.79 91.36 0.64 60.08 61.84 163.04 225.08

68,052.20 24.48 275.00 12.68 186.80 72.68 212.60 285.20

46,935.04 0.77 85.68 0.69 59.44 61.92 163.64 225.52

91,772.80 25.17 279.80 13.12 197.60 73.24 215.96 289.08

62,524.00 0.75 79.80 0.66 56.84 61.80 164.36 226.12

114,476.80 25.12 296.00 13.06 192.80 73.08 215.68 288.76

78,159.64 0.68 88.84 0.65 58.20 61.88 164.72 226.52

Figure 10. Comparison of the average makespan between the EDR and MWA TAOpt.

Figure 11. Comparison of the average idle times of the two shuttles between the EDR and MWA TAOpt.

EDR and MWA TAOpt. The results show that MWA TAOpt provides an average spare capacity of up to 400 pallets per working day compared to the reference target of 1440 pallets

and an improvement of up to 70% compared to the old reference (1080 pallets) of the original first-come, first-served dispatching rule currently used in the real plant. P



Article

Figure 12. Comparison of the maximum idle times of the two shuttles between the EDR and MWA TAOpt.

Figure 13. Comparison of the average execution times, the average waiting times and the average lead times of the pallet removal jobs between the EDR and MWA TAOpt.

optimal but close to optimal solutions within very limited amounts of computation time. In addition to the above results, a notable argument for deploying the proposed approach in practice is that for all the test instances solved by the reactive solution approach using TA models, the maximum computational time taken was less than 0.9 CPU seconds in all scheduling windows.

9. CONCLUSIONS AND OUTLOOK In this paper, a dynamic scheduling solution for problems under uncertainty which is based upon embedding TA-based schedule optimization in a moving window approach has been presented. The TA-based modeling framework that facilitates a modular approach and the solution by reachability analysis that computes feasible solutions very quickly are exploited to successfully handle online information on the arrival times and pallet routes within the challenging environment of the highrise warehouse shuttle system. The generic modeling framework of TA that can be used to model a wide range of different scheduling problems and the periodic rescheduling strategy of the moving window approach that explicitly considers updated information on dynamic, stochastic problem environments in later optimization windows make the proposed solution approach a suitable candidate for solving general dynamic scheduling problems. The performance of the approach was investigated for a series of problem instances using the TA-based scheduling tool

Figure 14. Comparison of the average load balance of the two shuttles between the EDR and MWA TAOpt.

Figure 15. Comparison of the maximum possible pallet removals per 16-h work day between the old reference, the target reference, the EDR and MWA TAOpt.

One of the common requirements in applying any sophisticated or rigorous mathematical approaches in industrial practice is that such approaches should deliver not necessarily Q



Article

In Proceedings of the 3rd Conference on Foundations of Computer-Aided Process Operations; CACHE Publications: MI, 1998; pp 91−111. (4) Shah, N. Single- and multisite planning and scheduling: Current status and future challenges. In Proceedings of the 3rd Conference on Foundations of Computer-Aided Process Operations; CACHE Publications: MI, 1998; pp 75−90. (5) Ierapetritou, M. G.; Floudas, C. A. Effective continuous-time formulation for short-term scheduling. 1. Multipurpose batch processes. Ind. Eng. Chem. Res. 1998, 37, 4341−4359. (6) Ierapetritou, M. G.; Floudas, C. A. Effective continuous-time formulation for short-term scheduling. 2. Continuous and semicontinuous processes. Ind. Eng. Chem. Res. 1998, 37, 4360−4374. (7) Ierapetritou, M. G.; Floudas, C. A. Short-term scheduling: New mathematical models vs. algorithmic improvements. Comput. Chem. Eng. 1998, 22, 419−426. (8) Kallrath, J. Combined strategic and operational planningan MILP success story in chemical industry. OR Spectrum 2002, 24, 315− 341. (9) Kallrath, J. Planning and scheduling in the process industry. OR Spectrum 2002, 24, 219−250. (10) Lim, M.; Karimi, I. A. Resource-constrained scheduling of parallel production lines using asynchronous slots. Ind. Eng. Chem. Res. 2003, 42, 6832−6842. (11) Maravelias, C. T.; Grossmann, I. E. New general continuoustime state task network formulation for short-term scheduling of multipurpose batch plants. Ind. Eng. Chem. Res. 2003, 42, 3056−3074. (12) Floudas, C. A.; Lin, X. Continuous-time versus discrete-time approaches for scheduling of chemical processes: A review. Comput. Chem. Eng. 2004, 28, 2109−2129. (13) Castro, P. M.; Barbosa-Povoa, A.; Matos, H.; Novais, A. Simple continuous-time formulation for short−term scheduling of batch and continuous processes. Ind. Eng. Chem. Res. 2004, 43, 105−118. (14) Sundaramoorthy, A.; Karimi, I. A. A simpler better slot-based continuous time formulation for scheduling in multipurpose batch plants. Chem. Eng. Sci. 2005, 60, 2697−2702. (15) Shaik, M. A.; Janak, S. L.; Floudas, C. A. Continuous-time models for short-term scheduling of multipurpose batch plants: A comparative study. Ind. Eng. Chem. Res. 2006, 45, 6190−6209. (16) Mendez, C. A.; Cerda, J.; Grossmann, I. E. State-of-the-art review of optimization methods for short-term scheduling of batch processes. Comput. Chem. Eng. 2006, 30, 913−946. (17) Zheng, J.; Zhou, X.; Ye, T. A review for short-term scheduling of batch processes. Proc. IEEE Int. Conf. Automation Logistics 2008, 1960−1964. (18) Maravelias, C. T. A general framework and modeling approach classification for chemical production scheduling. AIChE J. 2012, 58, 1812−1828. (19) Sand, G.; Engell, S. Modelling and solving real-time scheduling problems by stochastic integer programming. Comput. Chem. Eng. 2004, 28, 1087−1103. (20) Till, J.; Sand, G.; Urselmann, M.; Engell, S. A hybrid evolutionary algorithm for solving two-stage stochastic integer programs in chemical batch scheduling. Comput. Chem. Eng. 2007, 31, 630−647. (21) Tometzki, T.; Engell, S. Risk Conscious Solution of Problems under Uncertainty by Hybrid Multi-objective Evolutionary Algorithms. Comput. Chem. Eng. 2010, 35, 2521−2539. (22) Cui, J.; Engell, S. Medium-term Planning of a Multiproduct Batch Plant under Evolving Multi-period Multi-uncertainty by Means of a Moving Horizon Strategy. Comput. Chem. Eng. 2010, 34, 598− 619. (23) Tometzki, T.; Engell, S. Systematic Initialization Techniques for Hybrid Evolutionary Algorithms for Solving Two-Stage Stochastic Mixed-Integer Programs. Trans. Evol. Comput. 2011, 15, 196−214. (24) Kanakamedala, K. B.; Reklaitis, G. V.; Venkatasubramanian, V. Reactive schedule modification in multipurpose batch chemical plants. Ind. Eng. Chem. Res. 1994, 33, 77−90.

TAOpt, and numerical results were presented. An enhanced first-come, first-served dispatching rule similar to the original rule currently used in the plant was also introduced to evaluate the performance of the proposed reactive solution approach for the series of test instances in a comparison study. The results show that the proposed approach is substantially more efficient than the improved dispatching rule in terms of the reduction of the overall makespan, the reduction of the idle times of the shuttles, and the reduction of the mean lead times of the pallet removals. The numerical tests for the optimization target test instance of 1440 pallets per 16-h working day showed that the proposed approach can provide handling time gains of 70% compared to the previous throughput of 1080 pallets per 16-h working day (TAOpt MWA worst case for 1440 pallets: 31.6 s per pallet; current solution for 1080 pallets: 53.33 s per pallet). The EDR is not able to cope with the desired throughput gain in any case (EDR best case for 1440 pallets: 43.97 s per pallet). We believe that the solution which we presented here is prototypical for similar situations: the moving window approach is adequate to cope with rapidly changing situations in scheduling problems where new data arrives continuously, and the TAbased solution technique can provide good solutions very fast. In the situation dealt with in this paper, the individual problems could be solved to optimality considerably faster than required, but if this is not feasible, the TA-based approach will still provide a valid (and, according to our experience close to optimal) schedule that meets all constraints after a predefined maximum computation time, as the search is only done over feasible solutions. By choosing the branching and the pruning rules of the search of the reachability tree, the solution approach can be tailored to the situation; for example, if the available computation time is considerably shorter than required for an exhaustive search, the emphasis can be put on obtaining complete solutions fast. Classical priority rules can be integrated to steer the search, which thenin the situation of lack of computation timebecomes a neighborhood search around a solution generated by heuristic rules that proceeds until the computation time limit is reached. Future work includes the extension of the proposed approach to a reactive scheduling solution by replacing the periodic rescheduling strategy with a hybrid rescheduling strategy to handle other types of uncertainties at the time they occur, e.g. machine breakdowns or uncertainties in processing durations, and the integration, focused on the information exchange, of the scheduling solution on the scheduling level with the lower level sequence control to increase the error detecting and handling capabilities.

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: {christian.schoppmeyer|s.engell}@bci.tu-dortmund.de. Notes

The authors declare no competing financial interest.

■

REFERENCES

(1) Reklaitis, G. Overview of scheduling and planning of batch operations. Batch Process. Syst. Eng. 1992, 143, 660−705. (2) Mockus, L.; Reklaits, G. V. Mathematical programming formulation for scheduling of batch operations based on non-uniform time discretization. Comput. Chem. Eng. 1997, 21, 1147−1156. (3) Pekny, J. F.; Reklaitis, G. V. Towards the convergence of theory and practice: a technology guide for scheduling/planning methodology. R



Article

scheduling problem with job release dates. J. Intelligent Manuf. 2013, 24, 763−774. (48) Gomes, M.; Barbosa-Povoa, A. P.; Novais, A. Q. Reactive scheduling in a make-to-order flexible job shop with re-entrant process and assembly: a mathematical programming approach. Int. J. Prod. Res. 2013, 51, 5120−5141. (49) Kopanos, G. M.; Pistikopoulos, E. N. Reactive Scheduling by a Multiparametric Programming Rolling Horizon Framework: A Case of a Network of Combined Heat and Power Units. Ind. Eng. Chem. Res. 2014, 53, 4366−4386. (50) Rouwenhorst, B.; Reuter, B.; Stockrahm, V.; van Houtum, G. J.; Mantel, R. J.; Zijm, W. H. M. Warehouse design and control: Framework and literature review. Eur. J. Oper. Res. 2000, 122, 515− 533. (51) de Koster, R.; Le-Duc, T.; Roddbergen, K. J. Design and control of warehouse order picking: A literature review. Eur. J. Oper. Res. 2007, 182, 481−501. (52) Haneyah, S. W. A.; Schutten, J. M. J.; Schuur, P. C.; Zijm, W. H. M. Generic planning and control of automated material handling systems: practical requirements versus existing theory. Comput. Ind. 2013, 64, 177−190. (53) Amato, F.; Basile, F.; Carbone, C.; Chiacchio, P. An approach to control automated warehouse systems. Control Eng. Pract. 2005, 13, 1223−1241. (54) Kung, Y.; Kobayashi, Y.; Higashi, T.; Sugi, M.; Ota, J. Order scheduling of multiple stacker cranes on common rails in an automated storage/retrieval system. Int. J. Prod. Res. 2014, 52, 1171−1187. (55) Alur, R.; Dill, D. L. A theory of timed automata. Theor. Comput. Sci. 1994, 126, 183−235. (56) Fehnker, A. Scheduling a steel plant with timed automata. Proc. 6th Int. Conf. Real-Time Comput. Syst. Appl. 1999, 280−286. (57) Abdeddaim, Y. Scheduling with Timed Automata. Dissertation, Institut National Polytechnique de Grenoble, CNRS-VERIMAG: France, 2002. (58) Abdeddaim, Y.; Maler, O. Job shop scheduling using timed automata. LNCS 2002, 2102, 478−492. (59) Behrmann, G.; Fehnker, A.; Hune, T. S.; Petterson, P.; Larsen, K. G.; Romijn, J. Efficient guiding towards cost-optimality in uppaal. LNCS 2001, 2031, 174−188. (60) Larsen, K.; Behrmann, G.; Brinksma, E.; Fehnker, A.; Hune, T.; Pettersson, P.; Romijn, J. As Cheap as Possible: Efficient Cost-Optimal Reachability for Priced Timed Automata. LNCS 2001, 2102, 493−505. (61) Behrmann, G.; Larsen, K. G.; Rasmussen, J. I. Optimal scheduling using priced timed automata. ACM Sigmetrics 2005, 32, 34−40. (62) Panek, S.; Stursberg, O.; Engell, S. Efficient synthesis of production schedules by optimization of timed automata. Control Eng. Pract. 2006, 14, 1183−1197. (63) Godefroid, P. Using Partial Orders to Improve Automatic Verification Methods. LNCS 1999, 531, 176−185. (64) Panek, S.; Engell, S.; Subbiah, S.; Stursberg, O. Scheduling of multiproduct batch plants based upon timed automata models. Comput. Chem. Eng. 2008, 32, 275−291. (65) Panek, S.; Stursberg, O.; Engell, S. Job-shop scheduling by combining reachability analysis with linear programming. 7th Int. IFAC Workshop on Discrete Event Systems; IFAC: Reims, 2004; pp 199−204. (66) Subbiah, S.; Schoppmeyer, C.; Engell, S. Efficient scheduling of batch plants using reachability tree search for timed automata with lower bound computations. Comput.-Aided Chem. Eng. 2011, 29, 930− 934. (67) Schoppmeyer, C.; Subbiah, S.; Bendiganahalli-Deveerappa, S.; Engell, S. Effective Batch Scheduling with Sequence-dependent Changeovers Using Reachability Analysis of Timed Automata Combined with Lower Bound Computations. In 14th IFAC Symposium on Information Control Problems in Manufacturing [online]; Borangiu, T., Dolgui, A., Dumitrache, I., Filip, F.-G., Eds.; International Federation of Automatic Control: 2012; pp 75−80; http://www.ifacpapersonline.net (accessed January 07, 2014).

(25) Vin, P.; Ierapetritou, M. A New Approach for Efficient Rescheduling of Multiproduct Batch Plants. Ind. Eng. Chem. Res. 2000, 39, 4228−4238. (26) Mendez, C. A.; Cerda, J. Dynamic scheduling in multiproduct batch plants. Comput. Chem. Eng. 2003, 27, 1247−1259. (27) Mendez, C. A.; Cerda, J. An MILP framework for batch reactive scheduling with limited discrete resources. Comput. Chem. Eng. 2004, 27, 1059−1068. (28) van den Heever, S. A.; Grossmann, I. E. A strategy for the integration of production planning and reactive scheduling in the optimization of a hydrogen supply network. Comput. Chem. Eng. 2003, 27, 1813−1839. (29) Janak, S. L.; Floudas, C. A. Production Scheduling of a LargeScale Industrial Batch Plant. II. Reactive Scheduling. Ind. Eng. Chem. Res. 2006, 45, 8253−8269. (30) Capon-Garcia, E.; Bojarski, A. D.; Espuna, A.; Puigjaner, L. Multiobjective Evolutionary Optimization of Batch Process Scheduling Under Environmental and Economic Concerns. AIChE J. 2013, 59, 429−444. (31) Della Corce, F.; Salassa, F.; T’kindt, V. A hybrid heuristic approach for single machine scheduling with release dates. Comput. Oper. Res. 2014, 45, 7−11. (32) Hegyhati, M.; Friedler, F. Combinatorial Algorithms of the SGraph Framework for Batch Scheduling. Ind. Eng. Chem. Res. 2011, 50, 5169−5174. (33) Graham, R. L.; Lawler, E. L.; Lenstra, J. K.; Rinnooy Kan, A. H. G. Optimization and Approximation in Deterministic Sequencing and Scheduling: A Survey. Ann. Discrete Math. 1979, 5, 287−326. (34) Pinedo, M. L. Planning and Scheduling in Manufacturing and Services; Springer: Dordrecht, Heidelberg, London, New York, 2009; pp 1−555. (35) Lozano, A. J.; Medaglia, A. L. Scheduling of parallel machines with sequence-dependent batches and product incompatibilities in an automotive glass facility. J. Scheduling 2013, 1−20. (36) Zhu, X.; Wilhelm, W. E. Scheduling and lot sizing with sequence-dependent setup: A literature review. IEE Trans. 2006, 38, 987−1007. (37) Lin, B. M. T.; Lin, F. C.; Lee, R. C. T. Two-machine flow-shop scheduling to minimize total late work. Eng. Optim. 2006, 38, 501− 509. (38) Herrmann, J. W.; Chung-Yee, L.; Snowdon, J. A classification of static scheduling problems. In Complexity in Numerical Optimization; Pardalos, P. M., Ed.; World Scientific Publishing Co.: Singapore, 1993; pp 203−253. (39) Vieira, G. E.; Herrmann, J. W.; Lin, E. Rescheduling Manufacturing Systems: A Framework of Strategies, Policies, and Methods. J. Scheduling 2003, 6, 39−62. (40) Sabuncuoglu, I.; Goren, S. Hedging Production Schedules Against Uncertainty in Manufacturing Environment with a Review of Robustness and Stability Research. Int. J. Comput. Integr. Manuf. 2009, 22, 138−157. (41) Herrmann, J. W., Ed. Handbook of Production Scheduling; Springer Science and Business Media Inc.: New York, 2006; pp 135− 148. (42) Ouelhadj, D.; Petrovic, S. A survey of dynamic scheduling in manufacturing systems. J. Scheduling 2009, 12, 417−431. (43) Church, L. K.; Uzsoy, R. Analysis of periodic and event-driven rescheduling policies in dynamic shops. Int. J. Comput. Integr. Manuf. 1992, 5, 153−163. (44) Muhlemann, A. P.; Lockett, A. G.; Farn, C.-K. Job shop scheduling heuristics and frequency of scheduling. Int. J. Prod. Res. 1982, 20, 227−241. (45) Bierwirth, C.; Mattfeld, D. C. Production scheduling and rescheduling with genetic algorithms. Evolut. Comput. 1999, 7, 1−17. (46) Lin, L.; Gen, M.; Liang, Y.; Ohno, K. A Hybrid EA for Reactive Job-shop Scheduling. Proc. Comput. Sci. 2012, 12, 110−115. (47) Nie, L.; Gao, L.; Li, P.; Li, X. A GEP-based reactive scheduling policies constructing approach for dynamic flexible job shop S



Article

(68) Panek, S.; Stursberg, O.; Engell, S. Scheduling and planning with timed automata. Comput.-Aided Chem. Eng. 2006, 21, 1973−1978. (69) Subbiah, S.; Engell, S. Timed automata models for batch scheduling with sequence-dependent changeovers. Comput.-Aided Chem. Eng. 2009, 27, 1515−1520. (70) Subbiah, S.; Schoppmeyer, C.; Engell, S. An intuitive and efficient approach to process scheduling with sequence-dependent changeovers using timed automata models. Ind. Eng. Chem. Res. 2011, 50, 5131−5152. (71) Schoppmeyer, C.; Subbiah, S.; Engell, S. Modeling and Solving Batch Scheduling Problems with Various Storage Policies and Operational Policies using Timed Automata. Comput.-Aided Chem. Eng. 2012, 31, 635−639. (72) Abdeddaim, Y.; Asarin, E.; Maler, O. Scheduling with timed automata. Theor. Comput. Sci. 2006, 354, 272−300. (73) Gu, J.; Goetschalckx, M.; McGinnis, L. F. Research on warehouse operation: A comprehensive review. Eur. J. Oper. Res. 2007, 177, 1−21. (74) Subbiah, S.; Tometzki, T.; Engell, S. Batch Scheduling With Intermediate Due Dates Using Timed Automata Models. Comput.Aided Chem. Eng. 2008, 25, 151−156. (75) Subbiah, S.; Schoppmeyer, C.; De La Fuente Valdès, J. M.; Sonntag, C.; Engell, S. Optimal management of shuttle robots in a high-rise warehouse using timed automata models. In Proceedings of the 7th IFAC Conference on Manufacturing Modelling, Management, and Control [Online]; Bakhtadze, N., Chernyshov, K., Dolgui, A., Lototsky, V., Eds.; International Federation of Automatic Control: 2013; pp 1358−1363; http://www.ifac-papersonline.net (accessed January 07, 2014). (76) Schoppmeyer, C.; Hüfner, M.; Subbiah, S.; Engell, S. Timed Automata Based Scheduling for a Miniature Pipeless Plant with Mobile Robots. Proc. IEEE Int. Conf. Control Appl. 2012, 240−245. (77) De La Fuente Valdes, J. M. Optimization of the Shuttle Robots Management in a SABIC High Rise Warehouse (HRW). Master Thesis, Technische Universität Dortmund: December 2011. (78) Visual C# Resources. http://msdn.microsoft.com/en-gb/ vstudio/hh341490 (accessed Mai 26, 2012). (79) DOT.NET Framework and DOT.NET SDKs. http://msdn. microsoft.com/en-gb/vstudio/aa496123 (accessed Mai 26, 2012).

T


Dynamic Scheduling of Shuttle Robots in the ... - ACS Publications

Recommend Documents