Comparing experiments in discounted problems

Ludovic Renou,Xavier Venel

2024-06-04

Abstract:This paper compares statistical experiments in discounted problems, ranging from the simplest ones where the state is fixed and the flow of information exogenous to more complex ones, where the decision-maker controls the flow of information or the state changes over time.

Theoretical Economics,Statistics Theory

What problem does this paper attempt to address?

Long-Run Impulse Control with Generalized Discounting

Damian Jelito,Łukasz Stettner

DOI: https://doi.org/10.1137/23m1582539

IF: 2.2

2024-03-06

SIAM Journal on Control and Optimization

Abstract:SIAM Journal on Control and Optimization, Volume 62, Issue 2, Page 853-876, April 2024. In this paper, we investigate the effects of applying generalized (nonexponential) discounting on a long-run impulse control problem for a Feller–Markov process. We show that the optimal value of the discounted problem is the same as the optimal value of its undiscounted version. Next, we prove that an optimal strategy for the undiscounted discrete-time functional is also optimal for the discrete-time discounted criterion and nearly optimal for the continuous-time discounted one. This shows that the discounted problem, being time-inconsistent in nature, admits a time-consistent solution. Also, instead of a complex time-dependent Bellman equation, one may consider its simpler time-independent version.

mathematics, applied,automation & control systems
Stochastic control via direct comparison

Xi-Ren Cao,De-Xin Wang,Tao Lu,Yifan Xu

DOI: https://doi.org/10.1007/s10626-010-0093-4

2010-01-01

Abstract:The standard approach to stochastic control is dynamic programming. In this paper, we introduce an alternative approach based on direct comparison of the performance of any two policies. This is achieved by modeling the state process as a continuous-time and continuous-state Markov process and applying the same ideas as for the discrete-time and discrete-state case. This approach is simple and intuitively clear; it applies to different problems with, finite and infinite horizons, discounted and long-run-average performance, continuous and jump diffusions, in the same way. Discounting is not needed when dealing with long-run average performance. The approach provides a unified framework for stochastic control and other optimization theory and methodologies, including Markov decision processes, perturbation analysis, and reinforcement learning.
Discounting the Past

Taylor Dohmen,Ashutosh Trivedi

DOI: https://doi.org/10.48550/arXiv.2102.06985

2021-10-20

Abstract:Stochastic games with discounted payoff, introduced by Shapley, model adversarial interactions in stochastic environments where two players try to optimize a discounted sum of rewards. In this model, long-term weights are geometrically attenuated based on the delay in their occurrence. We propose a temporally dual notion -- called past-discounting -- where agents have geometrically decaying memory of the rewards encountered during a play of the game. We study objective functions based on past-discounted weight sequences and examine the corresponding stochastic games with liminf, discounted, and mean payoffs. For objectives specified as the limit inferior of past-discounted reward sequences, we show that positional determinacy fails and that optimal strategies may require unbounded memory. To overcome this obstacle, we study an approximate windowed objective based on the idea of using sliding windows of finite length to examine infinite plays. On the other hand, for objectives specified as the discounted and average limits of past-discounted reward sequences we establish determinacy in mixed stationary strategies in the setting of concurrent stochastic games and show how the values of these games may be computed via reductions to standard discounted and mean-payoff games.

Computer Science and Game Theory,Optimization and Control
Common Value Experimentation.

Jan Eeckhout,Xi Weng

DOI: https://doi.org/10.1016/j.jet.2015.10.002

IF: 1.79

2015-01-01

Journal of Economic Theory

Abstract:In many economic environments, agents often continue to learn about the same underlying state variable, even if they switch action. For example, a worker's ability revealed in one job or when unemployed is informative about her productivity in another job. We analyze a general setup of experimentation with common values, and show that in addition to the well-known conditions of value matching (level) and smooth pasting (first derivative), this implies that the second derivatives of the value function must be equal whenever the agent switches action. This condition holds generally whenever the stochastic process has continuous increments. The main appeal of our approach is its applicability, which is demonstrated with two applications featuring common value experimentation: strategic pricing, and job search with switching costs.
Experimenting under Stochastic Congestion

Shuangning Li,Ramesh Johari,Xu Kuang,Stefan Wager

2024-10-22

Abstract:We study randomized experiments in a service system when stochastic congestion can arise from temporarily limited supply or excess demand. Such congestion gives rise to cross-unit interference between the waiting customers, and analytic strategies that do not account for this interference may be biased. In current practice, one of the most widely used ways to address stochastic congestion is to use switchback experiments that alternatively turn a target intervention on and off for the whole system. We find, however, that under a queueing model for stochastic congestion, the standard way of analyzing switchbacks is inefficient, and that estimators that leverage the queueing model can be materially more accurate. Additionally, we show how the queueing model enables estimation of total policy gradients from unit-level randomized experiments, thus giving practitioners an alternative experimental approach they can use without needing to pre-commit to a fixed switchback length before data collection.

Systems and Control,Optimization and Control,Methodology
On Some Implications of Backward Discounting

Debraj Ray,Ruqu Wang

2001-01-01

Abstract:We study a model in which lifetime individual felicities are derived from bothpresent and past consumption streams. Eachof th ese streams is discounted, th e former forward in the usual way, the latter backward. This discounting method is time- inconsistent, and implies constant regrets by the individual. We assume that an individ- ual at date t evaluates consumption programs according to some weighted average of his own felicity (as perceived at date t) and that of a "future self" at some date T> t. This simple model shows that individuals may exhibit impatience across alternatives that are positioned in periods adjacent to the present, but patience across similar choices posi- tioned in the more distant future, that such impatience is attenuated as an individual grows older, and that lifetime choice plans are generally time-inconsistent. The model can also be used to capture the notion of parental influence and investigate its impact on savings. We finally examine other applications of "backward discounting" in the paper.
A formal analysis of inconsistent decisions in intertemporal choice through subjective time perception

Salvador Cruz Rambaud,Javier Sánchez García

DOI: https://doi.org/10.1016/j.heliyon.2023.e21077

IF: 3.776

2023-10-27

Heliyon

Abstract:The framework of this paper is subjective time perception in the context of intertemporal choice, that is to say, the process of making decisions on dated outcomes (monetary or not) by an individual or a group of individuals. In this setting, the Discounted Utility model and, more specifically, the exponential discounting have been the paradigmatic methodology used to measure the preferences on delayed outcomes. However, this model can only be applied to consistent choices in which individuals do not change their preferences when the involved rewards are delayed the same time interval. Unfortunately, this is not the case of several decision scenarios where time is viewed as a subjective variable. The objective of this paper is to formally analyze the consistency of intertemporal choices governed by a discount function, derived from the exponential, where time has been distorted according to certain psychological traits of the subjects involved in the decision-making. More specifically, the different types of decreasing impatience will be characterized by focusing on the distortion derived from the subjective perspective of time. The findings of this research are very relevant in order to explain the time-related behavior of decision-makers in some noteworthy fields such as finance, psychology, marketing or sociology.
Switching Costs in Frequently Repeated Games

Barton L. Lipman,Ruqu Wang

DOI: https://doi.org/10.1006/jeth.2000.2655

IF: 1.79

2000-01-01

Journal of Economic Theory

Abstract:We add small costs of changing actions and frequent repetition to finitely repeated games, making some surprising commitments credible. Naturally, switching costs make it credible not to change action. However, this can occur for small switching costs and gives a unique subgame perfect equilibrium in coordination games when Pareto dominance and risk dominance coincide. In the Prisoners' Dilemma, switching costs reduce the incentive to deviate from mutual cooperation, but reduce the incentive to switch from cooperation to punish defection. Hence whether switching costs enable cooperation depends on which effect dominates. Switching costs can make complex threats credible enabling a player to earn more than his Stackelberg payoff. Journal of Economic Literature Classification Numbers: C7, C73.
Switching Costs in Infinitely Repeated Games

Barton L. Lipman,Ruqu Wang

DOI: https://doi.org/10.1016/j.geb.2008.04.018

IF: 1.265

2009-01-01

Games and Economic Behavior

Abstract:We show that small switching costs can have surprisingly dramatic effects in infinitely repeated games if these costs are large relative to payoffs in a single period. This shows that the results in Lipman and Wang do have analogs in the case of infinitely repeated games [Lipman, B., Wang, R., 2000. Switching costs in frequently repeated games. J. Econ. Theory 93, August 2000, 149–190]. We also discuss whether the results here or those in Lipman–Wang imply a discontinuity in the equilibrium outcome correspondence with respect to small switching costs. We conclude that there is not a discontinuity with respect to switching costs but that the switching costs do create a discontinuity with respect to the length of a period.
The Contrast Effect in Temporal and Probabilistic Discounting

Cheng Chen,Guibing He

DOI: https://doi.org/10.3389/fpsyg.2016.00304

IF: 3.8

2016-01-01

Frontiers in Psychology

Abstract:In this information age, messages related to time, and uncertainty surround us. At the same time, our daily lives are filled with decisions accompanied by temporal delay or uncertainty. Will such information influence our temporal and probabilistic discounting? The authors address this question from the perspectives of decision by sampling (DbS) theory and psychological distance theory. Studies 1 and 2 investigated the effect of contextual messages on temporal discounting and probabilistic discounting, respectively. The results indicated that participants who memorized messages about long-term and low-probability events rated delay or uncertainty as mentally closer and exhibited a less degree of value discounting than those who memorized messages regarding short-term and high-probability events. In addition, a sense of distance from present or reality mediated the effect of contextual messages on value discounting. The implications of the current findings for theory and applications are discussed.
Using a Satisficing Model of Experimenter Decision-Making to Guide Finite-Sample Inference for Compromised Experiments

James J Heckman,Ganesh Karapakula

DOI: https://doi.org/10.1093/ectj/utab009

Abstract:This paper presents a simple decision-theoretic economic approach for analyzing social experiments with compromised random assignment protocols that are only partially documented. We model administratively constrained experimenters who satisfice in seeking covariate balance. We develop design-based small-sample hypothesis tests that use worst-case (least favorable) randomization null distributions. Our approach accommodates a variety of compromised experiments, including imperfectly documented re-randomization designs. To make our analysis concrete, we focus much of our discussion on the influential Perry Preschool Project. We reexamine previous estimates of program effectiveness using our methods. The choice of how to model reassignment vitally affects inference.
Probability discounting of environmental gains: do we multiply or add up?

Jiaxin Chen,Chuqian Chen,Yuqing Wang,Guibing He

DOI: https://doi.org/10.1080/13669877.2018.1485167

2019-01-01

Journal of Risk Research

Abstract:Individuals often discount and transform risky environmental outcomes into psychological certainty equivalences (CEs) during environmental decision-making. Their preference for different alternatives is influenced by the discounting degree. Our first experiment undertook a preliminary inspection of the probability discounting of environmental gains through a matching method; whereas in Experiment 2, the discounting rule of environmental gain was compared with monetary gain by a new line-projective method, which asked participants to project their subjective value evaluation explicitly on a line of specific length. The environmental gain and its mental value-equivalent monetary gain were analogical with respect to discounting degree and process model fitting. The probability discounting of both gains was better described by an additive-utility model than the normative exponential and hyperbolic models, which means that individuals were likely to discount the risky outcome by simply adding the disutility of uncertainty to the gain's nominal utility, rather than by multiplying the initial value by a discounting factor. It may be of great value in shedding light on the lay public evaluation of risky environmental interventions and boosting pro-environmental policy support.
Comparative Statics for Optimal Stopping Problems in Nonstationary Environments

Théo Durandard,Matteo Camboni

2024-07-31

Abstract:How do decisions change with the economic environment and with time? This paper studies general nonstationary stopping problems and provides the methodological tools to answer these questions. First, we identify conditions that ensure a monotone relation between decisions' timing and outcomes. These conditions apply to a prevalent class of economic environments. Second, we develop a theory of monotone comparative statics for stopping problems, offering general and unifying qualitative insights into the decision-maker's value and stopping behavior. We apply our results to models of information acquisition, bankruptcy, irreversible investment, and option pricing to explain documented patterns at odds with current theories.

Theoretical Economics,Analysis of PDEs,Optimization and Control
Profit-Driven Experimental Design

Yuhao Wang,Weiming Zhu

DOI: https://doi.org/10.2139/ssrn.3896229

2021-01-01

SSRN Electronic Journal

Abstract:From intense competition to the recent pandemic, companies face considerable volatility in the business environment. For companies that design experiments to identify parameters of interest and make subsequent policy decisions based on these parameters, the cost of such experimentation has become increasingly comparable to the economic gains obtained, as the insights offered by an experiment can be short-lived due to changing market conditions. In this paper, we develop a general framework to quantify the total expected profit from both the experimental and postexperimental stages given an experimental strategy. The proposed framework is constructed using the asymptotic properties of the underlying parameter estimates as a channel to connect the profits from the two stages. We demonstrate that the order of the optimal sample size and its regret are critically shaped by the curvature of the postexperimental profit function, which is defined as a quantitative measure of the local sensitivity of the postexperimental profit function to the parameter of interest. By exploiting this property, we are able to identify the order of the optimal sample size and its regret without prior knowledge. We illustrate through demand-learning newsvendor and pricing problems that the curvature function, the order of the optimal sample size and its regret can be derived in closed-form, despite that the postexperimental profit function may not be fully specified. We also develop an algorithm to numerically compute the order of the optimal sample size when the curvature function cannot be solved in closed-form. Finally, when prior information is available, we provide the lower and upper bounds of the optimal sample size that maximizes the total expected profit.
Long Run Stochastic Control Problems with General Discounting

Łukasz Stettner,Stettner, Łukasz

DOI: https://doi.org/10.1007/s00245-024-10118-5

2024-03-27

Applied Mathematics & Optimization

Abstract:Controlled discrete time Markov processes are studied first with long run general discounting functional. It is shown that optimal strategies for average reward per unit time problem are also optimal for average generally discounting functional. Then long run risk sensitive reward functional with general discounting is considered. When risk factor is positive then optimal value of such reward functional is dominated by the reward functional corresponding to the long run risk sensitive control. In the case of negative risk factor we get an asymptotical result, which says that optimal average reward per unit time control is nearly optimal for long run risk sensitive reward functional with general discounting, assuming that risk factor is close to 0. For this purpose we show in Appendix upper estimates for large deviations of weighted empirical measures, which are of independent interest.

mathematics, applied
Tradeoffs and Comparison Complexity

Cassidy Shubatt,Jeffrey Yang

2024-10-09

Abstract:This paper develops a theory of how tradeoffs govern comparison complexity, and how this complexity generates systematic mistakes in choice. In our model, options are easier to compare when they involve less pronounced tradeoffs, in particular when they are 1) more similar feature-by-feature and 2) closer to dominance. These two postulates yield tractable measures of comparison complexity in the domains of multiattribute, lottery, and intertemporal choice. We show how our model organizes a range of behavioral regularities in choice and valuation, such as context effects, preference reversals, and apparent probability weighting and hyperbolic discounting. We test our model experimentally by varying the strength and nature of tradeoffs. First, we show that our complexity measures predict choice errors, choice inconsistency, and cognitive uncertainty in binary choice data across all three domains. Second, we document that manipulations of comparison complexity can reverse classic behavioral regularities, in line with the predictions of the theory. We apply our theory to study strategic obfuscation by firms in a pricing game.

General Economics
On the Comparison of Discounted-Sum Automata with Multiple Discount Factors

Udi Boker,Guy Hefetz

DOI: https://doi.org/10.1007/978-3-031-30829-1_18

2023-06-09

Abstract:We look into the problems of comparing nondeterministic discounted-sum automata on finite and infinite words. That is, the problems of checking for automata $A$ and $B$ whether or not it holds that for all words $w$, $A(w)=B(w), A(w) \leq B(w)$, or $A(w)<B(w)$. These problems are known to be decidable when both automata have the same single integral discount factor, while decidability is open in all other settings: when the single discount factor is a non-integral rational; when each automaton can have multiple discount factors; and even when each has a single integral discount factor, but the two are different. We show that it is undecidable to compare discounted-sum automata with multiple discount factors, even if all are integrals, while it is decidable to compare them if each has a single, possibly different, integral discount factor. To this end, we also provide algorithms to check for given nondeterministic automaton $N$ and deterministic automaton $D$, each with a single, possibly different, rational discount factor, whether or not $N(w) = D(w)$, $N(w) \geq D(w)$, or $N(w) > D(w)$ for all words $w$.

Formal Languages and Automata Theory
Waiting in intertemporal choice tasks affects discounting and subjective time perception.

Ping Xu,Claudia González-Vallejo,Benjamin T. Vincent

DOI: https://doi.org/10.1037/xge0000771

2020-12-01

Abstract:The literature on human delay discounting behavior is dominated by experimental paradigms, which do not impose actual delays. Given that waiting may be aversive even on short timescales, we present a novel delay discounting paradigm to study differences in delay discounting behavior either when real waiting is involved, or not. This paradigm retains the fundamental trade-off between rewards received versus their immediacy. We used hierarchical Bayesian modeling to decompose and test models that separate discounting and subjective time perception mechanisms. We report 2 experiments that also explore the magnitude effect and gain-loss asymmetry. In both experiments, we found greater discounting and nonlinear (convex) time perception in the online waiting task, which required waiting after each choice, compared to a control condition where waiting was deferred until the end of the experiment. Discounting was also measured by a parameter of a hyperbolic-type model and related to reversals of preference between initial desire to wait and then not doing so. These results from our novel paradigm support the emerging view that subjective time perception plays an important role in intertemporal choice in addition to discounting tendencies. (PsycInfo Database Record (c) 2020 APA, all rights reserved)
An Empirical Study on Intertemporal Decision Making Under Risk

Martin Ahlbrecht,Martin Weber

DOI: https://doi.org/10.1287/mnsc.43.6.813

IF: 5.4

1997-06-01

Management Science

Abstract:This study compares time preference in the cases of certainty and risk. We analyze both matching and choice behavior. We find that violations of the stationarity axiom are restricted to matching behavior, both for certainty and risk. We also compare the discounting of certain and risky outcomes as well as the discounting of gains and losses. In matching tasks, certain outcomes are discounted more than risky ones. We could not confirm these results in a choice task. Gains and losses are not found to be discounted at different rates.

management,operations research & management science
A Comparison of Methods for Adaptive Experimentation

Samantha Horn,Sabina J. Sloman

DOI: https://doi.org/10.48550/arXiv.2207.00683

2022-07-02

Abstract:We use a simulation study to compare three methods for adaptive experimentation: Thompson sampling, Tempered Thompson sampling, and Exploration sampling. We gauge the performance of each in terms of social welfare and estimation accuracy, and as a function of the number of experimental waves. We further construct a set of novel "hybrid" loss measures to identify which methods are optimal for researchers pursuing a combination of experimental aims. Our main results are: 1) the relative performance of Thompson sampling depends on the number of experimental waves, 2) Tempered Thompson sampling uniquely distributes losses across multiple experimental aims, and 3) in most cases, Exploration sampling performs similarly to random assignment.

Methodology,Econometrics

Comparing experiments in discounted problems

Long-Run Impulse Control with Generalized Discounting

Stochastic control via direct comparison

Discounting the Past

Common Value Experimentation.

Experimenting under Stochastic Congestion

On Some Implications of Backward Discounting

A formal analysis of inconsistent decisions in intertemporal choice through subjective time perception

Switching Costs in Frequently Repeated Games

Switching Costs in Infinitely Repeated Games

The Contrast Effect in Temporal and Probabilistic Discounting

Using a Satisficing Model of Experimenter Decision-Making to Guide Finite-Sample Inference for Compromised Experiments

Probability discounting of environmental gains: do we multiply or add up?

Comparative Statics for Optimal Stopping Problems in Nonstationary Environments

Profit-Driven Experimental Design

Long Run Stochastic Control Problems with General Discounting

Tradeoffs and Comparison Complexity

On the Comparison of Discounted-Sum Automata with Multiple Discount Factors

Waiting in intertemporal choice tasks affects discounting and subjective time perception.

An Empirical Study on Intertemporal Decision Making Under Risk

A Comparison of Methods for Adaptive Experimentation