In metal additive manufacturing (AM) process, the complex thermal history, large temperature gradients and high cooling rates can induce high residual stresses and unique dislocation structures in the materials. The generation and evolution of these phenomena in AM process still remain incompletely explained due to experimental difficulties. In our work, a multiscale modeling approach is developed to investigate the thermal and residual stresses formation at grain scale during laser powder-bed fusion process, where the temperature profiles from powder-scale thermal-fluid flow simulations and the resultant polycrystalline structure from phase-field grain growth simulations are implemented into a thermal-mechanical crystal plasticity finite element model. This framework favors an observation of microscale residual stress evolution and characteristics in AM materials. Furthermore, the thermally induced deformation and temperature histories in specific grains can be extracted as the input of dislocation dynamics simulations. Then, a continuous dislocation dynamics (CDD) model is implemented to a dislocation-based crystal plasticity framework to study the dislocation dynamics under thermal stress in representative grains. Simulation results reveal the dislocation structure formation, evolution and stabilization induced by thermal stress in grains during laser scanning and cooling stages. Overall, this simulation framework can provide a powerful modeling tool to assist in the microscale mechanical analysis of AM materials and support the multiscale process-structure-property investigations in the future.