
Long-press the word, and you will receive a prompt asking whether you want to remove the word: To remove an existing word start typing the word until it appears in the suggestions list. Yes, this can be annoying, because it will automatically add mistakes and partial words from interrupted typing, including if your screen locks out due to inactivity. Once you've typed a word that's not recognized and accepted it (by hitting the space bar or enter, selecting it from the list of word suggestions, or exiting the application or screen where you're typing it), the word will be remembered. Instead, you can edit the dictionary directly from the keyboard.

For some reason I don't understand, the dictionary for Samsung Keyboard can't be accessed through settings. For example, for Swype Keyboard, you go to Settings→Languaguage and input, select the gear icon next to Swype in the Keyboards and input methods list, select My Words, then select Edit my dictionary. For most keyboards, you can access the user dictionary from the keyboard's own settings.

In stock Android, there's a User Dictionary selection available in Settings→Language and Input, which allows you to edit the user dictionary. You have this tagged as google-keyboard, but based on how you phrased the question I'm going to infer that you're using Samsung Keyboard, the default on Galaxy devices.
