The EL Theorem

Samuel Epstein

2023-09-10

Abstract:The combined universal probability $\mathbf{m}(D)$ of strings $x$ in sets $D$ is close to max $\mathbf{m}(x)$ over $x$ in $D$: their logs differ by at most $D$'s information $\mathbf{I}(D:\mathcal{H})$ about the halting sequence $\mathcal{H}$.

Computational Complexity,Logic

What problem does this paper attempt to address?

LIL and the Approximation of Rectangular Sums of B-valued Random Variables when Extreme Terms Are Excluded

Li Xin Zhang

DOI: https://doi.org/10.1007/s10114-002-0181-y

2002-01-01

Acta Mathematica Sinica English Series

Abstract:Let {X,X n ̄; n̄ ∈ ℕ d } be a field of i.i.d. random variables indexed by d-tuples of positive integers and taking values in a Banach space ${\user1{\mathcal{B}}}$ and let $X^{{{\left( r \right)}}}_{{\ifmmode\expandafter\bar\else\expandafter\=\fi{n}}} = X_{{\ifmmode\expandafter\bar\else\expandafter\=\fi{m}}} \;{\text{if}}\;{\left\| {X_{{\ifmmode\expandafter\bar\else\expandafter\=\fi{m}}} } \right\|}$ is the r-th maximum of {∥X k ̄∥ ; k̄ ≤ n̄}. Let S n ̄ =∑ k̄ ≤ n X k̄ and ${}^{{{\left( r \right)}}}S_{{\ifmmode\expandafter\bar\else\expandafter\=\fi{n}}} = S_{{\ifmmode\expandafter\bar\else\expandafter\=\fi{n}}} - {\left( {X^{{{\left( 1 \right)}}}_{{\ifmmode\expandafter\bar\else\expandafter\=\fi{n}}} + \cdots + X^{{{\left( r \right)}}}_{{\ifmmode\expandafter\bar\else\expandafter\=\fi{n}}} } \right)}$. We approximate the trimmed sums (r) S n ̄ by a Brownian sheet and obtain sufficient and necessary conditions for (r) S n to satisfy the compact and functional laws of the iterated logarithm. These results improve the previous works by Morrow (1981), Li and Wu (1989) and Ledoux and Talagrand (1990).
Generalized Stein's lemma and asymptotic equipartition property for subalgebra entropies

Li Gao,Mizanur Rahaman

2024-07-13

Abstract:The quantum Stein's lemma is a fundamental result of quantum hypothesis testing in the context of distinguishing two quantum states. A recent conjecture, known as the ``generalized quantum Stein's lemma", asserts that this result is true in a general framework where one of the states is replaced by convex sets of quantum states. In this work, we show that the assertion of the generalized Stein's lemma is true for the setting where the second hypothesis is the state space of any subalgebra $\mathcal{N}$. This is obtained through a strong asymptotic equipartition property for smooth subalgebra entropies that applies for any fixed smoothing parameter $\epsilon\in (0,1)$. As an application in resource theory, we show that the relative entropy of a subalgebra is the asymptotic dilution cost under suitable operations. This provides a scope to establish a connection between different quantum resources.

Quantum Physics,Mathematical Physics,Functional Analysis,Operator Algebras
The Ehrenfeucht conjecture: a compactness claim for finitely generated free monoids

Juhani Karhumäki

DOI: https://doi.org/10.1016/0304-3975(84)90004-5

IF: 1.002

1984-01-01

Theoretical Computer Science

Abstract:We survey recent results on the so-called Ehrenfeucht Conjecture which states: For each language L over a finite alphabet Σ there exists a finite subset F of L such that for each pair (g, h) of morphisms on Σ∗ the equation g(x)=h(x) holds for all x in L if and only if holds for all x in F.We point out that the conjecture is closely related to the theory of equations in free monoids. We also state a suprising consequence of the conjecture: If it holds (even noneffectively) for all DOL languages, then the HDOL sequence equivalence problem is decidable. Furthermore, we give examples of when the conjecture is known to hold. In particular, we establish it for all binary languages, as well as for all languages when attention is restricted to bounded delay morphisms of some fixed delay.

computer science, theory & methods
A correspondence between the time and space complexity

Ivan V. Latkin

2023-11-02

Abstract:We investigate the correspondence between the time and space recognition complexity of languages; for this purpose, we will code the long-continued computations of deterministic two-tape Turing machines by the relatively short-length quantified Boolean formulae. The modified Stockmeyer and Meyer method will appreciably be used for this simulation. It will be proved using this modeling that the complexity classes $\mathbf{EXP}$ and $\mathbf{PSPACE}$ coincide; and more generally, the class $(k\!+\!1)$-fold Deterministic Exponential Time equals to the class $k$-fold Deterministic Exponential Space for each $k\geqslant1$; the space complexity of the languages of the class $\mathbf{P}$ will also be studied. Furthermore, this allows us to slightly improve the early founded lower complexity bound of decidable theories that are nontrivial relative to some equivalence relation (this relation may be equality) -- each of these theories is consistent with the formula, which asserts that there are two non-equivalent elements. Keywords: computational complexity, the coding of computations through formulae, exponential time, polynomial space, lower complexity bound of the language recognition

Computational Complexity,Logic in Computer Science,Logic
On the computational power of $C$-random strings

Alexey Milovanov

DOI: https://doi.org/10.48550/arXiv.2409.04448

2024-08-24

Abstract:Denote by $H$ the Halting problem. Let $R_U: = \{ x | C_U(x) \ge |x|\}$, where $C_U(x)$ is the plain Kolmogorov complexity of $x$ under a universal decompressor $U$. We prove that there exists a universal $U$ such that $H \in P^{R_U}$, solving the problem posted by Eric Allender.

Computational Complexity
On Kolmogorov Structure Functions

Samuel Epstein

2024-07-10

Abstract:All strings with low mutual information with the halting sequence will have flat Kolmogorov Structure Functions, in the context of Algorithmic Statistics. Assuming the Independence Postulate, strings with non-negligible information with the halting sequence are purely mathematical constructions, and cannot be found in nature. Thus Algorithmic Statistics does not study strings in the physical world. This leads to the general thesis that two part codes require limitations as shown in the Minimum Description Length Principle. We also discuss issues with set-restricted Kolmogorov Structure Functions.

Computational Complexity
A Note on the Complexity of Comparing Succinctly Represented Integers, with an Application to Maximum Probability Parsing

Kousha Etessami,Alistair Stewart,Mihalis Yannakakis

DOI: https://doi.org/10.1145/2601327

2014-05-01

ACM Transactions on Computation Theory

Abstract:The following two decision problems capture the complexity of comparing integers or rationals that are succinctly represented in product-of-exponentials notation, or equivalently, via arithmetic circuits using only multiplication and division gates, and integer inputs. Input instance: Four lists of positive integers: a 1,..., an ∈N+ n ; b 1,..., bn ∈N+ n ; c 1,..., cm ∈N+ m ; d 1, ..., dm ∈N+ m ; where each of the integers is represented in binary. Problem 1 (equality testing): Decide whether a 1 b 1 a 2 b 2⋯ anbn = c 1 d 1 c 2 d 2⋯ cmdm . Problem 2 (inequality testing): Decide whether a 1 b 1 a 2 b 2⋯ anbn ≥ c 1 d 1 c 2 d 2⋯ cmdm . Problem 1 is easily decidable in polynomial time using a simple iterative algorithm. Problem 2 is much harder. We observe that the complexity of Problem 2 is intimately connected to deep conjectures and results in number theory. In particular, if a refined form of the ABC conjecture formulated by Baker in 1998 holds, or if the older Lang-Waldschmidt conjecture (formulated in 1978) on linear forms in logarithms holds, then Problem 2 is decidable in P-time (in the standard Turing model of computation). Moreover, it follows from the best available quantitative bounds on linear forms in logarithms, namely, by Baker and Wüstholz [1993] or Matveev [2000], that if m and n are fixed universal constants then Problem 2 is decidable in P-time (without relying on any conjectures). This latter fact was observed earlier by Shub [1993]. We describe one application: P-time maximum probability parsing for arbitrary stochastic context-free grammars (where ε -rules are allowed).
An Elementary Proof of the Inequality $χ\leq χ^*$ for Conditional Free Entropy

David Jekel,Jennifer Pi

2024-04-02

Abstract:Through the study of large deviations theory for matrix Brownian motion, Biane-Capitaine-Guionnet proved the inequality $\chi(X) \leq \chi^*(X)$ that relates two analogs of entropy in free probability defined by Voiculescu. We give a new proof of $\chi \leq \chi^*$ that is elementary in the sense that it does not rely on stochastic differential equations and large deviation theory. Moreover, we generalize the result to conditional microstates and non-microstates free entropy.

Operator Algebras,Probability
Chernoff's bound forms

M. Grendar Jr.,M. Grendar

DOI: https://doi.org/10.1063/1.1570535

2003-06-23

Abstract:Chernoff's bound binds a tail probability (ie. $Pr(X \ge a)$, where $a \ge EX$). Assuming that the distribution of $X$ is $Q$, the logarithm of the bound is known to be equal to the value of relative entropy (or minus Kullback-Leibler distance) for $I$-projection $\hat P$ of $Q$ on a set $\mathcal{H} \triangleq \{P: E_PX = a\}$. Here, Chernoff's bound is related to Maximum Likelihood on exponential form and consequently implications for the notion of complementarity are discussed. Moreover, a novel form of the bound is proposed, which expresses the value of the Chernoff's bound directly in terms of the $I$-projection (or generalized $I$-projection).

Probability,Statistics Theory
On the theorem of Davenport and generalized Dedekind sums

Bence Borda

DOI: https://doi.org/10.48550/arXiv.1606.07946

2016-06-26

Abstract:A symmetrized lattice of $2n$ points in terms of an irrational real number $\alpha$ is considered in the unit square, as in the theorem of Davenport. If $\alpha$ is a quadratic irrational, the square of the $L^2$ discrepancy is found to be $c( \alpha ) \log n + O \left( \log \log n \right)$ for a computable positive constant $c( \alpha )$. For the golden ratio $\varphi$, the value $\sqrt{c (\varphi ) \log n}$ yields the smallest $L^2$ discrepancy of any sequence of explicitly constructed finite point sets in the unit square. If the partial quotients $a_k$ of $\alpha$ grow at most polynomially fast, the $L^2$ discrepancy is found in terms of $a_k$ up to an explicitly bounded error term. It is also shown that certain generalized Dedekind sums can be approximated using the same methods. For a special generalized Dedekind sum with arguments $a, b$ an asymptotic formula in terms of the partial quotients of $\frac{a}{b}$ is proved.

Number Theory
A solution of the generalised quantum Stein's lemma

Ludovico Lami

2024-10-23

Abstract:We solve the generalised quantum Stein's lemma, proving that the Stein exponent associated with entanglement testing, namely, the quantum hypothesis testing task of distinguishing between $n$ copies of an entangled state $\rho_{AB}$ and a generic separable state $\sigma_{A^n:B^n}$, equals the regularised relative entropy of entanglement. Not only does this determine the ultimate performance of entanglement testing, but it also establishes the reversibility of all quantum resource theories under asymptotically resource non-generating operations, with the regularised relative entropy of resource governing the asymptotic transformation rate between any two quantum states. As a by-product, we prove that the same Stein exponent can also be achieved when the null hypothesis is only approximately i.i.d., in the sense that it can be modelled by an "almost power state". To solve the problem we introduce two techniques. The first is a procedure that we call "blurring", which, informally, transforms a permutationally symmetric state by making it more evenly spread across nearby type classes. Blurring alone suffices to prove the generalised Stein's lemma in the fully classical case, but not in the quantum case. Our second technical innovation, therefore, is to perform a second quantisation step to lift the problem to an infinite-dimensional bosonic quantum system; we then solve it there by using techniques from continuous-variable quantum information. Rather remarkably, the second-quantised action of the blurring map corresponds to a pure loss channel. A careful examination of this second quantisation step is the core of our quantum solution.

Quantum Physics,Mathematical Physics
Derandomization under Different Resource Constraints

Samuel Epstein

2023-09-12

Abstract:We provide another proof to the EL Theorem. We show the tradeoff between compressibility of codebooks and their communication capacity. A resource bounded version of the EL Theorem is proven. This is used to prove three instances of resource bounded derandomization. This paper is in support of the general claim that if the existence of an object can be proven with the probabilistic method, then bounds on its Kolmogorov complexity can be proven as well.

Computational Complexity,Information Theory
A uniform Dvoretzky-Kiefer-Wolfowitz inequality

Daniel Bartl,Shahar Mendelson

2023-12-11

Abstract:We show that under minimal assumption on a class of functions $\mathcal{H}$ defined on a probability space $(\mathcal{X},\mu)$, there is a threshold $\Delta_0$ satisfying the following: for every $\Delta\geq\Delta_0$, with probability at least $1-2\exp(-c\Delta m)$ with respect to $\mu^{\otimes m}$, \[ \sup_{h\in\mathcal{H}} \sup_{t\in\mathbb{R}} \left| \mathbb{P}(h(X)\leq t) - \frac{1}{m}\sum_{i=1}^m 1_{(-\infty,t]}(h(X_i)) \right| \leq \sqrt{\Delta};\] here $X$ is distributed according to $\mu$ and $(X_i)_{i=1}^m$ are independent copies of $X$. The value of $\Delta_0$ is determined by an unexpected complexity parameter of the class $\mathcal{H}$ that captures the set's geometry (Talagrand's $\gamma_1$-functional). The bound, the probability estimate and the value of $\Delta_0$ are all optimal up to a logarithmic factor.

Probability
On the Convergence of the ELBO to Entropy Sums

Jörg Lücke,Jan Warnken

2024-04-29

Abstract:The variational lower bound (a.k.a. ELBO or free energy) is the central objective for many established as well as many novel algorithms for unsupervised learning. During learning such algorithms change model parameters to increase the variational lower bound. Learning usually proceeds until parameters have converged to values close to a stationary point of the learning dynamics. In this purely theoretical contribution, we show that (for a very large class of generative models) the variational lower bound is at all stationary points of learning equal to a sum of entropies. For standard machine learning models with one set of latents and one set of observed variables, the sum consists of three entropies: (A) the (average) entropy of the variational distributions, (B) the negative entropy of the model's prior distribution, and (C) the (expected) negative entropy of the observable distribution. The obtained result applies under realistic conditions including: finite numbers of data points, at any stationary point (including saddle points) and for any family of (well behaved) variational distributions. The class of generative models for which we show the equality to entropy sums contains many well-known generative models. As concrete examples we discuss Sigmoid Belief Networks, probabilistic PCA and (Gaussian and non-Gaussian) mixture models. The result also applies for standard (Gaussian) variational autoencoders, a special case that has been shown previously (Damm et al., 2023). The prerequisites we use to show equality to entropy sums are relatively mild. Concretely, the distributions of a given generative model have to be of the exponential family, and the model has to satisfy a parameterization criterion (which is usually fulfilled). Proving the equality of the ELBO to entropy sums at stationary points (under the stated conditions) is the main contribution of this work.

Machine Learning,Information Theory,Probability,Statistics Theory
Entropy Concentration and the Empirical Coding Game

Peter Grunwald

DOI: https://doi.org/10.48550/arXiv.0809.1017

2008-09-05

Abstract:We give a characterization of Maximum Entropy/Minimum Relative Entropy inference by providing two `strong entropy concentration' theorems. These theorems unify and generalize Jaynes' `concentration phenomenon' and Van Campenhout and Cover's `conditional limit theorem'. The theorems characterize exactly in what sense a prior distribution Q conditioned on a given constraint, and the distribution P, minimizing the relative entropy D(P ||Q) over all distributions satisfying the constraint, are `close' to each other. We then apply our theorems to establish the relationship between entropy concentration and a game-theoretic characterization of Maximum Entropy Inference due to Topsoe and others.

Information Theory,Machine Learning,Statistics Theory,Methodology
Elastic-Degenerate String Comparison

Esteban Gabory,Moses Njagi Mwaniki,Nadia Pisanti,Solon P. Pissis,Jakub Radoszewski,Michelle Sweering,Wiktor Zuba

2024-11-12

Abstract:An elastic-degenerate (ED) string $T$ is a sequence of $n$ sets $T[1],\ldots,T[n]$ containing $m$ strings in total whose cumulative length is $N$. We call $n$, $m$, and $N$ the length, the cardinality and the size of $T$, respectively. The language of $T$ is defined as $L(T)=\{S_1 \cdots S_n\,:\,S_i \in T[i]$ for all $i\in[1,n]\}$. ED strings have been introduced to represent a set of closely-related DNA sequences, also known as a pangenome. The basic question we investigate here is: Given two ED strings, how fast can we check whether the two languages they represent have a nonempty intersection? We call the underlying problem the ED String Intersection (EDSI) <a class="link-external link-http" href="http://problem.For" rel="external noopener nofollow">this http URL</a> two ED strings $T_1$ and $T_2$ of lengths $n_1$ and $n_2$, cardinalities $m_1$ and $m_2$, and sizes $N_1$ and $N_2$, respectively, we show the following: - There is no $O((N_1N_2)^{1-\epsilon})$-time algorithm, for any constant $\epsilon>0$, for EDSI even when $T_1$ and $T_2$ are over a binary alphabet, unless the Strong Exponential-Time Hypothesis is false. - There is no combinatorial $O((N_1+N_2)^{1.2-\epsilon}f(n_1,n_2))$-time algorithm, for any constant $\epsilon>0$ and any function $f$, for EDSI even when $T_1$ and $T_2$ are over a binary alphabet, unless the Boolean Matrix Multiplication conjecture is false. - An $O(N_1\log N_1\log n_1+N_2\log N_2\log n_2)$-time algorithm for outputting a compact (RLE) representation of the intersection language of two unary ED strings. In the case when $T_1$ and $T_2$ are given in a compact representation, we show that the problem is NP-complete. - An $O(N_1m_2+N_2m_1)$-time algorithm for EDSI. - An $\tilde{O}(N_1^{\omega-1}n_2+N_2^{\omega-1}n_1)$-time algorithm for EDSI, where $\omega$ is the exponent of matrix multiplication; the $\tilde{O}$ notation suppresses factors that are polylogarithmic in the input size.

Data Structures and Algorithms
Noncommutative Donoho-Stark-Elad-Bruckstein-Ricaud-Torrésani Uncertainty Principle

K. Mahesh Krishna

2024-08-09

Abstract:Let $\{\tau_n\}_{n=1}^\infty$ and $\{\omega_m\}_{m=1}^\infty$ be two modular Parseval frames for a Hilbert C*-module $\mathcal{E}$. Then for every $x \in \mathcal{E}\setminus\{0\}$, we show that \begin{align} (1) \quad \quad \quad \quad \|\theta_\tau x \|_0 \|\theta_\omega x \|_0 \geq \frac{1}{\sup_{n, m \in \mathbb{N}} \|\langle \tau_n, \omega_m\rangle \|^2}. \end{align} We call Inequality (1) as \textbf{Noncommutative Donoho-Stark-Elad-Bruckstein-Ricaud-Torrésani Uncertainty Principle}. Inequality (1) is the noncommutative analogue of breakthrough Ricaud-Torrésani uncertainty principle \textit{[IEEE Trans. Inform. Theory, 2013]}. In particular, Inequality (1) extends Elad-Bruckstein uncertainty principle \textit{[IEEE Trans. Inform. Theory, 2002]} and Donoho-Stark uncertainty principle \textit{[SIAM J. Appl. Math., 1989]}.

Operator Algebras,Functional Analysis
On the Gap between Hereditary Discrepancy and the Determinant Lower Bound

Lily Li,Aleksandar Nikolov

2024-01-17

Abstract:The determinant lower bound of Lovasz, Spencer, and Vesztergombi [European Journal of Combinatorics, 1986] is a powerful general way to prove lower bounds on the hereditary discrepancy of a set system. In their paper, Lovasz, Spencer, and Vesztergombi asked if hereditary discrepancy can also be bounded from above by a function of the hereditary discrepancy. This was answered in the negative by Hoffman, and the largest known multiplicative gap between the two quantities for a set system of $m$ substes of a universe of size $n$ is on the order of $\max\{\log n, \sqrt{\log m}\}$. On the other hand, building on work of Matoušek [Proceedings of the AMS, 2013], recently Jiang and Reis [SOSA, 2022] showed that this gap is always bounded up to constants by $\sqrt{\log(m)\log(n)}$. This is tight when $m$ is polynomial in $n$, but leaves open what happens for large $m$. We show that the bound of Jiang and Reis is tight for nearly the entire range of $m$. Our proof relies on a technique of amplifying discrepancy via taking Kronecker products, and on discrepancy lower bounds for a set system derived from the discrete Haar basis.

Combinatorics,Discrete Mathematics
D numbers theory: a generalization of Dempster-Shafer evidence theory

Yong Deng

DOI: https://doi.org/10.48550/arXiv.1405.3175

2014-05-13

Abstract:Efficient modeling of uncertain information in real world is still an open issue. Dempster-Shafer evidence theory is one of the most commonly used methods. However, the Dempster-Shafer evidence theory has the assumption that the hypothesis in the framework of discernment is exclusive of each other. This condition can be violated in real applications, especially in linguistic decision making since the linguistic variables are not exclusive of each others essentially. In this paper, a new theory, called as D numbers theory (DNT), is systematically developed to address this issue. The combination rule of two D numbers is presented. An coefficient is defined to measure the exclusive degree among the hypotheses in the framework of discernment. The combination rule of two D numbers is presented. If the exclusive coefficient is one which means that the hypothesis in the framework of discernment is exclusive of each other totally, the D combination is degenerated as the classical Dempster combination rule. Finally, a linguistic variables transformation of D numbers is presented to make a decision. A numerical example on linguistic evidential decision making is used to illustrate the efficiency of the proposed D numbers theory.

Artificial Intelligence
On some congruences and exponential sums

Moubariz Z. Garaev,Igor E. Shparlinski

2023-04-18

Abstract:Let $\varepsilon>0$ be a fixed small constant, ${\mathbb F}_p$ be the finite field of $p$ elements for prime $p$. We consider additive and multiplicative problems in ${\mathbb F}_p$ that involve intervals and arbitrary sets. Representative examples of our results are as follows. Let ${\mathcal M}$ be an arbitrary subset of ${\mathbb F}_p$. If $\#{\mathcal M} >p^{1/3+\varepsilon}$ and $H\ge p^{2/3}$ or if $\#{\mathcal M} >p^{3/5+\varepsilon}$ and $H\ge p^{3/5+\varepsilon}$ then all, but $O(p^{1-\delta})$ elements of ${\mathbb F}_p$ can be represented in the form $hm$ with $h\in [1, H]$ and $m\in {\mathcal M}$, where $\delta> 0$ depends only on $\varepsilon$. Furthermore, let $ {\mathcal X}$ be an arbitrary interval of length $H$ and $s$ be a fixed positive integer. If $$ H> p^{17/35+\varepsilon}, \quad \#{\mathcal M} > p^{17/35+\varepsilon}. $$ then the number $T_6(\lambda)$ of solutions of the congruence $$ \frac{m_1}{x_1^s}+ \frac{m_2}{x_2^s}+ \frac{m_3}{x_3^s}+\frac{m_4}{x_4^s}+ \frac{m_5}{x_5^s}+\frac{m_6}{x_6^s} \equiv \lambda\mod p, \qquad m_i\in {\mathcal M}, \quad \ x_i \in {\mathcal X}, \quad i =1, \ldots, 6, $$ satisfies $$ T_6(\lambda)=\frac{H^6(\#{\mathcal M})^6}{p}\left(1+O(p^{-\delta})\right), $$ where $\delta> 0$ depends only on $s$ and $\varepsilon$.

Number Theory

The EL Theorem

LIL and the Approximation of Rectangular Sums of B-valued Random Variables when Extreme Terms Are Excluded

Generalized Stein's lemma and asymptotic equipartition property for subalgebra entropies

The Ehrenfeucht conjecture: a compactness claim for finitely generated free monoids

A correspondence between the time and space complexity

On the computational power of $C$-random strings

On Kolmogorov Structure Functions

A Note on the Complexity of Comparing Succinctly Represented Integers, with an Application to Maximum Probability Parsing

An Elementary Proof of the Inequality $χ\leq χ^*$ for Conditional Free Entropy

Chernoff's bound forms

On the theorem of Davenport and generalized Dedekind sums

A solution of the generalised quantum Stein's lemma

Derandomization under Different Resource Constraints

A uniform Dvoretzky-Kiefer-Wolfowitz inequality

On the Convergence of the ELBO to Entropy Sums

Entropy Concentration and the Empirical Coding Game

Elastic-Degenerate String Comparison

Noncommutative Donoho-Stark-Elad-Bruckstein-Ricaud-Torrésani Uncertainty Principle

On the Gap between Hereditary Discrepancy and the Determinant Lower Bound

D numbers theory: a generalization of Dempster-Shafer evidence theory

On some congruences and exponential sums