Connect with SSH to your machine and execute next command

sudo dpkg-reconfigure locales

After this, you will see a list of languages and countries. You have to select with space bar the language and country you need. In my case, as I live in Spain, I have selected es_ES.UTF-8. It is recommended to choose UTF-8 version.

Finally, Ubuntu will ask you to select the default language to use in the operating system.

After that, you should configure the language in the keyboard with this command:

sudo dpkg-reconfigure keyboard-configuration

Now, we will configure the timezone

sudo dpkg-reconfigure tzdata

I have select the country (Europe) and the city (Madrid) where I live, so the timezone has been updated.

Finally it is recommended to reboot the system to apply the changes.

sudo reboot

Once rebooted, you can check the language has been updated.

# locale
LANG=es_ES.UTF-8
LANGUAGE=
LC_CTYPE="es_ES.UTF-8"
LC_NUMERIC="es_ES.UTF-8"
LC_TIME="es_ES.UTF-8"
LC_COLLATE="es_ES.UTF-8"
LC_MONETARY="es_ES.UTF-8"
LC_MESSAGES="es_ES.UTF-8"
LC_PAPER="es_ES.UTF-8"
LC_NAME="es_ES.UTF-8"
LC_ADDRESS="es_ES.UTF-8"
LC_TELEPHONE="es_ES.UTF-8"
LC_MEASUREMENT="es_ES.UTF-8"
LC_IDENTIFICATION="es_ES.UTF-8"
LC_ALL=

And as it was expected, the language es_ES.UTF-8 is defined as default.