Data Compression with Prime Numbers

Gordon Chalmers

DOI: https://doi.org/10.48550/arXiv.physics/0511145

2005-11-16

General Physics

Abstract:A compression algorithm is presented that uses the set of prime numbers. Sequences of numbers are correlated with the prime numbers, and labeled with the integers. The algorithm can be iterated on data sets, generating factors of doubles on the compression.

What problem does this paper attempt to address?

Parallel Data Compression Techniques

David Noel,Elizabeth Graham,Liyuan Liu

2023-06-20

Abstract:With endless amounts of data and very limited bandwidth, fast data compression is one solution for the growing datasharing problem. Compression helps lower transfer times and save memory, but if the compression takes too long, this no longer seems viable. Multi-core processors enable parallel data compression; however, parallelizing the algorithms is anything but straightforward since compression is inherently serial. This paper explores techniques to parallelize three compression schemes: Huffman coding, LZSS, and MP3 coding

Distributed, Parallel, and Cluster Computing
Information loss and run time from practical application of quantum data compression

Saahil Patel,Benjamin Collis,William Duong,Daniel Koch,Massimiliano Cutugno,Laura Wessing,Paul Alsing

DOI: https://doi.org/10.1088/1402-4896/acc492

2023-03-17

Physica Scripta

Abstract:We examine information loss, resource costs, and run time from practical application of quantum data compression. Compressing quantum data to fewer qubits enables efficient use of resources, as well as applications for quantum communication and denoising. In this context, we provide a description of the quantum and classical components of the hybrid quantum autoencoder algorithm, implemented using IBM's Qiskit language. Utilizing our own data sets, we encode bitmap images as quantum superposition states, which correspond to linearly independent vectors with density matrices of discrete values. We successfully compress this data with near-lossless compression using simulation, and then run our algorithm on an IBMQ quantum chip. We describe conditions and run times for training and compressing our data on quantum devices, and relate trainability to specific characteristics and performance metrics of our parametric quantum circuits.

physics, multidisciplinary
A New Technique for Text Data Compression

Udita N. Katugampola

DOI: https://doi.org/10.1109/IS3C.2012.109

2010-12-20

Abstract:In this paper we use ternary representation of numbers for compressing text data. We use a binary map for ternary digits and introduce a way to use the binary 11-pair, which has never been use for coding data before, and we futher use 4-Digits ternary representation of alphabet with lowercase and uppercase with some extra symbols that are most commonly used in day to day life. We find a way to minimize the length of the bits string, which is only possible in ternary representation thus drastically reducing the length of the code. We also find some connection between this technique of coding dat and Fibonacci numbers.

Cryptography and Security,Information Theory
Prime Numbers, Dirichlet Density, and Benford's Law

Alex Ely Kossovsky

DOI: https://doi.org/10.48550/arXiv.1603.08501

2016-03-08

Abstract:The Prime Numbers are well-known for their paradoxical stand regarding Benford's Law. On one hand they adamantly refuse to obey the law of Benford in the usual sense, namely that of a normal density of the proportion of primes with d as the leading digit, yet on the other hand, the Dirichlet density for the subset of all primes with d as the leading digit is indeed LOG(1 + 1/d). In this article the superficiality of the Dirichlet density result is demonstrated and explained in terms of other well-known and established results in the discipline of Benford's Law, conceptually concluding that prime numbers cannot be considered Benford at all, in spite of the Dirichlet density result. In addition, a detailed examination of the digital behavior of prime numbers is outlined, showing a distinct digital development pattern, from a slight preference for low digits at the start for small primes, to a complete digital equality for large primes in the limit as the prime number sequence goes to infinity. Finally an exact analytical expression for the density of the logarithms of primes is derived and shown to be always on the rise at the macro level, an observation that is also confirmed empirically.

General Mathematics
The Compression method and applications

Theophilus Agama

2024-04-08

Abstract:In this paper we introduce and develop the method of compression of points in space. We introduce the notion of the mass, the rank, the entropy, the cover and the energy of compression. We leverage this method to prove some class of inequalities related to Diophantine equations. In particular, we show that for each $L<n-1$ and for each $K>n-1$, there exist some $(x_1,x_2,\ldots,x_n)\in \mathbb{N}^n$ with $x_i\neq x_j$ for all $1\leq i<j\leq n$ such that \begin{align}\frac{1}{K^{n}}\ll \prod \limits_{j=1}^{n}\frac{1}{x_j}\ll \frac{\log (\frac{n}{L})}{nL^{n-1}}\nonumber \end{align}and that for each $L>n-1$ there exist some $(x_1,x_2,\ldots,x_n)$ with $x_i\neq x_j$ for all $1\leq i<j\leq n$ and some $s\geq 2$ such that \begin{align}\sum \limits_{j=1}^{n}\frac{1}{x_j^s}\gg s\frac{n}{L^{s-1}}.\nonumber \end{align}

Number Theory
Spatio-temporal chaos and patterns formation in nonequilibrium media: phenomenological model of electronic turbulence

E.S.Mchedlova,D.I.Trubetskov

DOI: https://doi.org/10.48550/arXiv.chao-dyn/9607010

1996-07-19

Abstract:The processes in nonequilibrium dissipative media caused by coherent structure formation and lead to the complicated dynamics are of interest for nonlinear physics. Here we consider a model of the flow of interacting electronics patterns. Under electronic pattern we understand a small volume of active media, containing electron oscillators. The superradiation phenomenon takes place due to the evolving radiation instability within each small volume of the flow. It is numerically shown that the dynamics of radiation is changed from spatio-temporal complex oscillations to arising of spatial disorder, i. e. chaotic spatial pattern formation.

Chaotic Dynamics
Fast Parallel DNA-Based Algorithms for Molecular Computation: Determining a Prime Number

Weng-Long Chang,Michael (Shan-Hui) Ho,Minyi Guo,Xiaohong Jiang,Jingling Xue,Minglu Li

DOI: https://doi.org/10.1109/ICITA.2005.145

2005-01-01

Abstract:This paper demonstrates that testing whether a natural number is a prime number or not can also be solved with O(k鲁) biological operations, O(2^k) DNA library strands, O(k虏) the longest length of DNA library and O(1) tubes, where k is the number of bits for a natural number to be checked.
A Novel Lossless Encoding Algorithm for Data Compression - Genomics Data as an Exemplar

Anas Al-okaily,Abdelghani Tbakhi

DOI: https://doi.org/10.1101/2020.08.24.264366

2024-01-11

Abstract:Data compression is a challenging and increasingly important problem. As the amount of data generated daily continues to increase, efficient transmission and storage has never been more critical. In this study, a novel encoding algorithm is proposed, motivated by the compression of DNA data and associated characteristics. The proposed algorithm follows a divide-and-conquer approach by scanning the whole genome, classifying subsequences based on similarity patterns, and binning similar subsequences together. The data are then compressed in each bin independently. This approach is different than the currently known approaches: entropy, dictionary, predictive, or transform based methods. Proof-of-concept performance was evaluated using a benchmark dataset with seventeen genomes ranging in size from kilobytes to gigabytes. The results showed considerable improvement in the compression of each genome, preserving several megabytes compared with state-of-art tools. Moreover, the algorithm can be applied to the compression of other data types include mainly text, numbers, images, audio, and video which are being generated daily and unprecedentedly in massive volumes.

Bioinformatics
Data Compression and the 8&#x00d7;8 Integer Transform

Min Zhu,Jin Wen,Leibo Liu,Shaojun Wei

DOI: https://doi.org/10.1109/asicon.2009.5351549

2009-01-01

Abstract:Data compression is a kind of power concentrating process. Complex data could be transferred into eigenvalues and sparse data concentrated directly. DCT is a very efficient transform but hard to implement. 8×8 integer transform derived here is easily mapping into a parallel accelerating system, and proved to earn a better performance than existing 4×4 ones. This work implemented 8×8 integer algorithm on the REmus system, a novel reconfigurable system designed by Tsinghua, verified the architecture with several media applications. A 46.0% speedup comparing with 4×4 integer ones and a 60.1% average boost comparing with other processors are achieved. Different dividing of frames also has an impact on other algorithms, such as deblocking, the most significant process in h264 codec. It gets 62.5% boost, too. Comparing with 4×4 ones, when quantity level is smaller than 10, 8×8 restores better pictures and a 10% improvement is always found on compress ratios.
Identification Codes via Prime Numbers

Emad Zinoghli,Mohammad Javad Salariseddigh

2024-08-22

Abstract:We introduce a method for construction of identification codes based on prime number generation over the noiseless channels. The earliest method for such construction based on prime numbers was proposed by Ahlswede which relies on algorithms for generation of prime numbers. This method requires knowledge of $2^n$ first prime numbers for identification codes with block length $n,$ which is not computationally efficient. In this work, we revisit Ahlswede's scheme and propose a number of modifications. In particular, employing probabilistic prime generation algorithm, we guarantee that the prime keys generation is possible in polynomial time. Furthermore, additional improvements in terms of type II upper bound are derived and presented. Finally, we propose a method for identification coding based on hash functions which generalizes the Ahlswede's scheme.

Information Theory
A special sequence and primorial numbers

Amit Kumar Basistha,Eugen J. Ionascu

2023-11-22

Abstract:In this paper, we study a class of functions defined recursively on the set of natural numbers in terms of the greatest common divisor algorithm of two numbers and requiring a minimality condition. These functions are permutations, products of infinitely many cycles that depend on certain breaks in the natural numbers involving the primes, and some special products of primes with a density of approximately $29.4\%$. We show that these functions split into only two equivalence classes (modulo the natural equivalence relation of eventually identical maps): one is the class of the identity map and the other is generated by a map whose discrete derivative is almost periodic with ``periods" the primorial numbers.

Number Theory
Modified HuffBit Compress Algorithm – An Application of R

Nahida Habib,Kawsar Ahmed,Iffat Jabin,Mohammad Motiur Rahman

DOI: https://doi.org/10.1515/jib-2017-0057

2018-02-22

Journal of Integrative Bioinformatics

Abstract:Abstract The databases of genomic sequences are growing at an explicative rate because of the increasing growth of living organisms. Compressing deoxyribonucleic acid (DNA) sequences is a momentous task as the databases are getting closest to its threshold. Various compression algorithms are developed for DNA sequence compression. An efficient DNA compression algorithm that works on both repetitive and non-repetitive sequences known as “HuffBit Compress” is based on the concept of Extended Binary Tree. In this paper, here is proposed and developed a modified version of “HuffBit Compress” algorithm to compress and decompress DNA sequences using the R language which will always give the Best Case of the compression ratio but it uses extra 6 bits to compress than best case of “HuffBit Compress” algorithm and can be named as the “Modified HuffBit Compress Algorithm”. The algorithm makes an extended binary tree based on the Huffman Codes and the maximum occurring bases (A, C, G, T). Experimenting with 6 sequences the proposed algorithm gives approximately 16.18 % improvement in compression ration over the “HuffBit Compress” algorithm and 11.12 % improvement in compression ration over the “2-Bits Encoding Method”.
The prime-counting Copeland–Erdős constant

J. M. Campbell

DOI: https://doi.org/10.1007/s10474-024-01437-8

2024-06-07

Acta Mathematica Hungarica

Abstract:Let denote a sequence of nonnegative integers. Let denote the real number obtained by concatenating the digit expansions, in a fixed base, of consecutive entries of . Research on digit expansions of this form has mainly to do with the normality of for a given base. Famously, the Copeland-Erdős constant , for the case whereby equals the prime number , is normal in base 10. However, it seems that the "inverse" construction given by concatenating the decimal digits of , where denotes the prime-counting function, has not previously been considered. Exploring the distribution of sequences of digits in this new constant would be comparatively difficult, since the number of times a fixed appears in is equal to the prime gap , with the behaviour of prime gaps notoriously elusive. Using a combinatorial method due to Szüsz and Volkmann, we prove that Cramér's conjecture on prime gaps implies the normality of in a given base , for .

mathematics
Integer Set Compression and Statistical Modeling

N. Jesper Larsson

DOI: https://doi.org/10.48550/arXiv.1402.1936

2014-02-09

Abstract:Compression of integer sets and sequences has been extensively studied for settings where elements follow a uniform probability distribution. In addition, methods exist that exploit clustering of elements in order to achieve higher compression performance. In this work, we address the case where enumeration of elements may be arbitrary or random, but where statistics is kept in order to estimate probabilities of elements. We present a recursive subset-size encoding method that is able to benefit from statistics, explore the effects of permuting the enumeration order based on element probabilities, and discuss general properties and possibilities for this class of compression problem.

Information Theory,Data Structures and Algorithms
A General Framework for Progressive Data Compression and Retrieval

Victor A. P. Magri,Peter Lindstrom

2023-08-08

Abstract:In scientific simulations, observations, and experiments, the cost of transferring data to and from disk and across networks has become a significant bottleneck that particularly impacts subsequent data analysis and visualization. To address this challenge, compression techniques have been widely adopted. However, traditional lossy compression approaches often require setting error tolerances conservatively to respect the numerical sensitivities of a wide variety of post hoc data analyses, some of which may not even be known a priori. Progressive data compression and retrieval has emerged as a solution, allowing for the adaptive handling of compressed data according to the needs of a given post-processing task. However, few analysis algorithms natively support progressive data processing, and adapting compression techniques, file formats, client/server frameworks, and APIs to support progressivity can be challenging. This work presents a general framework that supports progressive-precision data queries independently of the underlying data compressor or number representation. Our approach is based on a multiple-component representation that successively, with each new component, reduces the error between the original and compressed field, allowing each field in the progressive sequence to be expressed as a partial sum of components. We have implemented our approach on top of four popular scientific data compressors and have evaluated its behavior on several real-world data sets from the SDRBench collection. Numerical results indicate that our framework is effective in terms of accuracy compared to each of the standalone compressors it builds upon. In addition, (de)compression time is proportional to the number and granularity of components. Finally, our framework allows for fully lossless compression using lossy compressors when a sufficient number of components are employed.

Databases,Graphics
Polynomial data compression for large-scale physics experiments

Pierre Aubert,Thomas Vuillaume,Gilles Maurin,Jean Jacquemier,Giovanni Lamanna,Nahid Emad

DOI: https://doi.org/10.48550/arXiv.1805.01844

2018-05-04

Abstract:The new generation research experiments will introduce huge data surge to a continuously increasing data production by current experiments. This data surge necessitates efficient compression techniques. These compression techniques must guarantee an optimum tradeoff between compression rate and the corresponding compression /decompression speed ratio without affecting the data integrity. This work presents a lossless compression algorithm to compress physics data generated by Astronomy, Astrophysics and Particle Physics experiments. The developed algorithms have been tuned and tested on a real use case~: the next generation ground-based high-energy gamma ray observatory, Cherenkov Telescope Array (CTA), requiring important compression performance. Stand-alone, the proposed compression method is very fast and reasonably efficient. Alternatively, applied as pre-compression algorithm, it can accelerate common methods like LZMA, keeping close performance.

Numerical Analysis,Instrumentation and Methods for Astrophysics,Computational Physics
Genbit Compress Tool(GBC): A Java-Based Tool to Compress DNA Sequences and Compute Compression Ratio(bits/base) of Genomes

P. Raja Rajeswari,Allam Apparo,V. K. Kumar,P.Raja Rajeswari,V.K. Kumar

DOI: https://doi.org/10.48550/arXiv.1006.1193

2010-06-07

Mathematical Software

Abstract:We present a Compression Tool, "GenBit Compress", for genetic sequences based on our new proposed "GenBit Compress Algorithm". Our Tool achieves the best compression ratios for Entire Genome (DNA sequences) . Significantly better compression results show that GenBit compress algorithm is the best among the remaining Genome compression algorithms for non-repetitive DNA sequences in Genomes. The standard Compression algorithms such as gzip or compress cannot compress DNA sequences but only expand them in size. In this paper we consider the problem of DNA compression. It is well known that one of the main features of DNA Sequences is that they contain substrings which are duplicated except for a few random Mutations. For this reason most DNA compressors work by searching and encoding approximate repeats. We depart from this strategy by searching and encoding only exact repeats. our proposed algorithm achieves the best compression ratio for DNA sequences for larger genome. As long as 8 lakh characters can be given as input While achieving the best compression ratios for DNA sequences, our new GenBit Compress program significantly improves the running time of all previous DNA compressors. Assigning binary bits for fragments of DNA sequence is also a unique concept introduced in this program for the first time in DNA compression.
The Fast Fibonacci Decompression Algorithm

R.Baca,V.Snasel,J.Platos,M.Kratky,E.El-Qawasmeh

DOI: https://doi.org/10.48550/arXiv.0712.0811

2007-12-19

Abstract:Data compression has been widely applied in many data processing areas. Compression methods use variable-size codes with the shorter codes assigned to symbols or groups of symbols that appear in the data frequently. Fibonacci coding, as a representative of these codes, is used for compressing small numbers. Time consumption of a decompression algorithm is not usually as important as the time of a compression algorithm. However, efficiency of the decompression may be a critical issue in some cases. For example, a real-time compression of tree data structures follows this issue. Tree's pages are decompressed during every reading from a secondary storage into the main memory. In this case, the efficiency of a decompression algorithm is extremely important. We have developed a Fast Fibonacci decompression for this purpose. Our approach is up to $3.5\times$ faster than the original implementation.

Performance,Other Computer Science
Infinitude of Palindromic Almost-Prime Numbers

Aleksandr Tuxanidy,Daniel Panario

DOI: https://doi.org/10.1093/imrn/rnae174

2024-08-08

International Mathematics Research Notices

Abstract:Abstract It is proven that, in any given base, there are infinitely many palindromic numbers having at most six prime divisors, each relatively large. The work involves equidistribution estimates for the palindromes in residue classes to large moduli, offering upper bounds for moments and averages of certain products closely related to exponential sums over palindromes.

mathematics
Time-universal data compression and prediction

Boris Ryabko

DOI: https://doi.org/10.1109/isit.2019.8849224

2019-07-01

Abstract:Suppose there is a large file which should be transmitted (or stored) and there are several (say, m) admissible data-compressors. It seems natural to try all the compressors and then choose the best, i.e. the one that gives the shortest compressed file. Then transfer (or store) the index number of the best compressor (it requires [log m] bits) the compressed file. The only problem is the time, which essentially increases due to the need to compress the file m times (in order to find the best compressor). We propose a method that encodes the file with the optimal compressor, but uses a relatively small additional time: the ratio of this extra time and the total time of calculation can be limited by an arbitrary positive constant. A similar situation occurs when forecasting time series. Generally speaking, in many situations it may be necessary find the best data compressor (or predictor) out of a given set, which is often done by comparing them empirically. One of the goals of this work is to turn such a selection process into a part of the data compression method, automating and optimizing it.

Data Compression with Prime Numbers

Parallel Data Compression Techniques

Information loss and run time from practical application of quantum data compression

A New Technique for Text Data Compression

Prime Numbers, Dirichlet Density, and Benford's Law

The Compression method and applications

Spatio-temporal chaos and patterns formation in nonequilibrium media: phenomenological model of electronic turbulence

Fast Parallel DNA-Based Algorithms for Molecular Computation: Determining a Prime Number

A Novel Lossless Encoding Algorithm for Data Compression - Genomics Data as an Exemplar

Data Compression and the 8&amp;#x00d7;8 Integer Transform

Identification Codes via Prime Numbers

A special sequence and primorial numbers

Modified HuffBit Compress Algorithm – An Application of R

The prime-counting Copeland–Erdős constant

Integer Set Compression and Statistical Modeling

A General Framework for Progressive Data Compression and Retrieval

Polynomial data compression for large-scale physics experiments

Genbit Compress Tool(GBC): A Java-Based Tool to Compress DNA Sequences and Compute Compression Ratio(bits/base) of Genomes

The Fast Fibonacci Decompression Algorithm

Infinitude of Palindromic Almost-Prime Numbers

Time-universal data compression and prediction

Data Compression and the 8×8 Integer Transform