Fit a wooden batten - decorate it and use wood screws to secure the brackets of the pole.
If you attempt to fix the brackets directly to the wall it will may be fine for perhaps six months maybe more but the fixings will loosen with use and you will end up having to put a batten up.
Best fixings for the batten are the type that come with a large screw and a plastic sleeve that you hammer in together - Hammerfix Frame Fixing or similar.
Peccavi, October 2008