Complex systems, influenced by relationships among random variables, are often characterized using the joint probability distribution function. Computing this $n_{th}$-order function can be highly challenging. A practical solution involves approximating it with lower-order functions by imposing constraints on the dependencies among random variables. This thesis investigates approximating Markov networks by maximizing the spanning k-tree (MSKT) on the network, a computationally intensive task. Nonetheless, the thesis presents a polynomial time algorithm to address this challenge, provided that the identified MSKT retains specific properties. These properties, often observed in practical applications, make the MSKT algorithm useful for statistical inferences with real-world data. Many NP-hard problems on general graphs can be solved in polynomial time when the input is a spanning k-tree. We present one application of this work to augment Naive Bayes Classifier. Naive Bayes classifiers assume all the features are independent given the class label. To represent dependency of features with lower order function we used MSKT algorithm.

Another objective is to address the challenges associated with modeling interdependence's among hidden events that have generated observable data. Traditional models struggle with the intractability arising from computing context-sensitive relations, which can compromise the quality of answers when decoding arbitrary structures. To overcome this issue, the thesis introduces the arbitrary order hidden Markov model a-hmm, an extension of the hidden Markov models, specifically designed to decode the optimal higher-order structure while ensuring computational tractability. The advantage of the a-hmm lies in its ability to leverage an identified principle on how random variables influence each other in a stochastic process. The thesis demonstrates the practical application of the decoding model by developing a simple yet effective algorithm for RNA secondary structure prediction.