Laser powder bed fusion (LPBF) is a popular additive manufacturing process for creating intricate metal components. However, this process is notorious for generating residual stresses that can cause distortion and cracking of the part, which poses a significant challenge to its mechanical performance and integrity. Thermomechanical simulation is an effective way to analyse and optimize LPBF process conditions to ensure the quality of the final product. The constitutive model used in these simulations, which defines the relationship between stresses and strains of the material, is a critical factor that affects the accuracy of the predictions. Despite its importance, there is a lack of guidance on choosing the appropriate constitutive model in the literature. In this work, we investigate the impact of using different constitutive formulations on the predicted residual stresses and distortions in LPBF of Hastelloy X (HX). We compare and evaluate various commonly used phenomenological models: classical elasto-plastic models with isotropic, kinematic, or combined hardening, and an elasto-viscoplastic model with combined hardening. We calibrate these models using outcomes of a set of cyclic isothermal strain-controlled tests across a wide range of temperatures for LPBF HX. To verify the predictive capabilities of the constructed thermomechanical models, we developed an experimental setup that uses strain gauges to measure in-situ distortion during LPBF on a miniature cantilever. We used a sequential coupled thermomechanical modeling framework to predict residual stress and distortion by considering different constitutive models. The thermal FE analysis was the same for all models and was previously validated using observations from operando X-ray diffraction analysis during the print over the cantilever. We found that the ability of the constitutive model to capture the Bauschinger effect and strain-rate sensitivity had the most significant impact on the predicted residual stress and distortion. These findings demonstrate the importance of selecting an appropriate constitutive model for numerical analysis of residual stress and distortion in LPBF processes.