Yes, that's the way I understand it. Harassment needs to either (or both) cross some of the significant lines immediately, or be part of a pattern. Basically, you can get in trouble either with a great big leap, or by consistent effort over time :-). What you pretty much can't do is get into bad trouble with one small isolated mistake.
[ETA "isolated"; you can and should be able to get into trouble with one small mistake if it's the same mistake you've been making over and over again for far too long.]