Abstract:The use of coordinate processes for the modelling of impulse control for general Markov processes typically involves the construction of a probability measure on a countable product of copies of the path space. In addition, admissibility of an impulse control policy requires that the random times of the interventions be stopping times with respect to different filtrations arising from the different component coordinate processes. When the underlying Markov process has continuous paths, however, a simpler model can be developed which takes the single path space as its probability space and uses the natural filtration with respect to which the intervention times must be stopping times. Moreover, this model construction allows for impulse control with random effects whereby the decision maker selects a distribution of the new state. This paper gives the construction of the probability measure on the path space for an admissible intervention policy subject to a randomized impulse mechanism. In addition, a class of polices is defined for which the paths between interventions are independent and a further subclass for which the cycles following the initial cycle are identically distributed. A benefit of this smaller subclass of policies is that one is allowed to use classical renewal arguments to analyze long-term average control problems. Further, the paper defines a class of {\em stationary}\/ impulse policies for which the family of models gives a Markov family. The decision to use an $(s,S)$ ordering policy in inventory management provides an example of an impulse policy for which the process has i.i.d.~cycles and the family of models forms a Markov family.

Continuous-Time Controlled Markov Chains

Continuous-Time Controlled Markov Chains with Discounted Rewards

Continuous-time Markov Chains Based H<inf>&#x221E;</inf> Control of Networked Control Systems

Constrained Continuous-Time Markov Control Processes with Discounted Criteria

Drift and Monotonicity Conditions for Continuous-Time Controlled Markov Chains with an Average Criterion.

Constrained Total Undiscounted Continuous-Time Markov Decision Processes

Constrained Continuous-Time Markov Decision Processes with Average Criteria

Continuous-Time Markov Decision Processes with State-Dependent Discount Factors

Denumerable Continuous-Time Markov Decision Processes with Multiconstraints on Average Costs

Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates

New Discount and Average Optimality Conditions for Continuous-Time Markov Decision Processes

Absorbing Continuous-Time Markov Decision Processes with Total Cost Criteria

Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates

Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates.

Total Reward Criteria for Unconstrained/constrained Continuous-Time Markov Decision Processes.

Continuous Time Markov Decision Processes with Nonuniformly Bounded Transition Rate: Expected Total Rewards

Continuous-Time Markov Decision Processes with Discounted Rewards: the Case of Polish Spaces

Denumerable-state Continuous-Time Markov Decision Processes with Unbounded Transition and Reward Rates under the Discounted Criterion

On the Modelling of Impulse Control with Random Effects for Continuous Markov Processes

Discounted Continuous-Time Constrained Markov Decision Processes in Polish Spaces

Markov Chains On Continuous State Space