So how should they approach GL...the jury is still out (by that I mean I really don't know

).
As far as convincing the audience that the new film is different and worth seeing...the only thing that would really do that imho is to either recast Hal or use a different GL, John Stewart in particular. With Stewart you can tap into a whole lot of people who watched the animated JLU cartoon and probably thought GL was black anyway. And while I don't know who they would cast you could potentially draw in a different audience depending on how popular of an actor they cast. While this could work with Guy or Kyle too, I think John brings a clearer message of this is different now for no other reason that him being black, and the number of superhero movies staring a black lead are few.
Aside from that just thinking about what worked and didn't work from the first film, ie what needs the most rehabilitation...I think it's Hal, the tone, character development, etc. None of which require a full on reboot. Also how exactly did people react to the film? Is it as bad as some on these boards say or was it more "meh...it was ok but not great" sort of deal, like the Fantastic Four or something similar? Or was there real outright, "I hate this movie" from the general audiences? B/c I saw that Ryan Reynolds won the kid's choice award for best superhero back when the film came out so some people must not have hated it. If the GA weren't impressed but didn't hate it, then I thiink you could move forward with a sequel but just focus on what will make the film stand out as different this time around and convince them tonally that it's worth their time and money.
If making a sequel I would focus on improving some of the designs. In some ways i think they alienated the audience by being too alien. Redesign Oa, so that it looks less abstract and more impressive, like asgard from Thor. You want people to be in awe of this place and enjoy the scenes there instead of being uncomfortable or uninterested. Make more practical sets this to ground the film and make those space scenes more interesting. Use practical suits, it will help sell the reality of the place and I think improve performances as well. I think those aesthetic changes would at least visually show the audience that "hey we listened and we've improved".
2nd focus on the tone. Make it more interesting. Give is a grand scifi space drama vibe instead of a superhero one. Plenty of successful scifi films took place off earth and weren't hurt by it. Why? Because they had well developed plots. GL was all over the place, unfocused and undeveloped. Really focus on his relationships in this scifi world. Make the aliens interesting but not repulsive. star wars and plenty of other scifi films have given us proper aliens who were also very likable and made you want to see more. Yet not a single alien GL seemed that interesting, other than Kilowag, Sinestro (who's basically human) and Tomar Re.
Also I think with the new GL cartoon is testing the public and building a receptive audience. Now you have proof that it's not GL but simply the the film iteration that didn't resonate with audiences. If they continue to test ideas with that cartoon, get a good feel of what people expect from GL and deliver that, then I think even if we don't a full on amazing film we'll at least get our version of Incredible Hulk. A film that delivered the good properly and setup the character for the future.