If you have outlook contacts saved onto your Android, make sure to import these contact BEFORE removing the account from the settings
Contacts > Options (3 dots) > Manage Contacts > Import/Export > Select Storage Location.
Once Outlook is configured, Follow the steps below to re-sync Outlook contacts on Android:
1. Open Outlook on your Android.
2. Tap your photo \ Office icon or the 3 lines on the Top-Left.
3. Tap the Cogwheel on the Bottom-Left.
4. This will bring you to the app settings, Tap on your Office 365 Account.
5. Turn on the Option for Sync Contacts.