Prince Harry is opening up about his relationship with his brother Prince William following the former's decision to step back from the royal family.
In a new clip from his sit-down interview with Oprah Winfrey, which aired Sunday on CBS, Harry and wife Meghan Markle further explained what prompted their royal exit and how the rest of the British monarchy reacted to the news.
For more on Prince Harry and Meghan Markle's interview with Oprah, listen below to the episode of PEOPLE Every Day.
While Meghan and Harry are no longer working royals, the prince said that he will still "always be there" for his brother William and his family in the interview portion that was released on CBS This Morning, Monday.
Asked whether or not Harry's family members reached out to apologize for the reasons he felt he had to leave, the 36-year-old responded, "No, sadly not."
"The feeling is that this was our decision therefore the consequence is on us. And despite three years of asking for help and visualizing how this might end, it was … it's been really hard," he explained. "I am part of the system with them, I always have been, but I guess — and I'm very aware of this — my brother can't leave that system but I have."
"With that relationship and that control and the fear by the U.K. tabloids, it's a toxic environment but I will always be there for [William] and I will always be there for my family," he added.
Paul Grover - WPA Pool/Getty Images Queen Elizabeth, Meghan Markle, Prince Harry, Prince William and Kate Middleton
Harry said that he believes his father, Prince Charles, has "had to make peace" with the so-called toxic environment.
As for why Harry and Meghan decided to step back instead of taking a similar approach, the Duchess of Sussex asserted that "this was different."
"[Social media] didn't exist, " she explained. "Plus my being American, it translated in a different way across the pond so you had a noise level that was very different."
"I think the volume of what was coming in and the interest was greater because of social media, because of the fact that I was not just British, and unfortunately if his family say 'Well this is what's happened to all of us,' if they can compare what the experience that I went through was similar to what has been shared with us ... this is not the same," she told Oprah.
Meghan added: "If a member of this family would comfortably say, 'We've all had to deal with things that are rude,' rude and racist are not the same. And equally, you've also had a press team that got on the record to defend you, especially when they know something is not true, and that did not happen for us."
In the initial interview footage that aired, Harry admitted that there is "a lot to work through" when it comes to his relationship with his father Prince Charles.
Can't get enough of PEOPLE's Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!
"I feel really let down, because he's been through something similar," Harry said of Charles. "He knows what pain feels like. I will always love him, but there is a lot of hurt that's happened. I will continue to make it one of my priorities to try to heal that relationship. But they only know what they know."
Of brother William, he added, "I love William to bits. He's my brother. We've been through hell together. But we are on different paths."
Asked by Oprah how he would describe the relationship now, he said, "the relationship is space. Time heals all things, hopefully."