According to ESPN, Cleveland's general manager David Griffin put all his focus into courting Kevin Durant during the 2016 summer. This urgency came from the growing possibility that KD would leave the Thunder and join the Warriors. And after narrowly defeating Golden State in a historic Finals performance, letting the Warriors add Durant to their superstar roster was not something Griffin was willing to do. As a result, he claimed he would deplete the entire roster excluding LeBron James to stop this from coming to fruition. Even then, Cleveland's meeting with KD seemed more like a formality rather than a real possibility.
"I don't believe you can dream big enough in the NBA," Griffin said in regards to the lofty idea of recruiting Durant. "You have to go through the exercise."
Durant's decision not to join LeBron in Cleveland might be have been driven by his desire to escape James' shadow. As noted, KD fully thought he would be considered the best player in the world after beating Cleveland in the 2017 NBA Finals. Although that series didn't make him the league's undisputed leader, if Durant had gone to the Cavs he wouldn't even have been in a position to tighten the race.
Much like Kyrie Irving, at this point KD was looking to establish his own elite status. This would not have been possible with the Cavs, even with a waning LeBron. Since 2003, Cleveland's success seemed to be dependent on James. This is even evident when James is not with the Cavs, as Cleveland quickly falls from title contender to lottery pick. That combined with the dramatic fashion in which 'Bron won the team their first NBA title makes him forever linked to the franchise. As a result, KD would essentially be "hustling backwards" by joining a franchise that's so connected with James.