Sometimes Oliver Hudson and Robyn Lively wish they were almost more famous. 

The Rules of Engagement alum and Teen Witch star have watched their sisters—Kate Hudson and Blake Lively, respectively—become Hollywood A-listers, which has afforded the actresses roles and opportunities that they themselves haven't had.

"There's always a part of me that's like, ‘F--k, man, why can't I do that?'" Oliver admitted to guest Robyn on the April 7 episode of his Sibling Rivalry podcast. "‘I want to do a role like that. I want to work with people like that. I want to make that kind of money. I don't want to be f--king hustling for everything.'" 

The 47-year-old clarified that his jealousy didn't mean he wasn't "happy" for his younger sister, whom he has so much "love" and "gratitude" for.

"But," he added, "I'm not ashamed to admit that I have envy."

In response, Robyn confessed that she "totally" understood his feelings, recalling times where she felt envious of Blake's career milestones and found herself "wishing it were me."

"In a healthy way, I do think, 'Oh, I would die to work with so-and-so,'" she admitted. "That I do, because I've been in this industry for so long."

Noting that no one in the Lively family expected Blake to achieve her level of celebrity because "she was so shy," Robyn explained that she's still "so proud" of her sister's achievements, stating, "Everything she's gotten and she has, she's earned."

But Blake's fame has created a level of public interest that not only impacts her and her husband Ryan Reynolds, but also their four kids: James, 9, Ines, 7, Betty, 4, and a fourth child whose birth they revealed in February 2023.  

"My sister's life is a lot," explained Robyn, who shares kids Baylen Johnson, 21, and Kate Johnson, 19, with husband Bart Johnson. "It's a lot of work to maintain that. It really is."

And ultimately, for her, the cons may outweigh the pros.

"It's a totally different world," Robyn continued. "I don't know if I could handle that, I really don't."  

Keep reading to find out more stars who come from famous families. 

It's family night for the Lopez gang as they celebrate the launch of the Disney Pixar 'Finding Dory' collection from Trina Turk in Manhattan Beach, Calif.

 

The Bachelorette star poses with her family of four in front of their brand-new évolur nursery. 

The actor and his sons spend the day at Knotts Berry Farm theme park ins Buena Park, Calif. where they meet up with Snoopy. 

The True Tori star has a blast with her four kids at Knott's Berry Farm in celebration of their new ride "Voyage to the Iron Reef 4D." 

The Real Housewives of Beverly Hills stars celebrate the Fourth of July in true red, white and blue fashion. 

It's Disney night at Madison Square Garden as the family of four watch Disney Live! Mickey's Music Festival in New York City.

We've got a very good feeling about this! The Craft Wars star throws a Star Wars themed-bash for Liam's birthday. 

The sisters spoil their kids with plenty of toys at Toys R Us after a delicious lunch in Burbank, Calif.

Awww! The sporty pair hang out on the sidelines during a soccer game in Encino, Calif. 

The former Disney star is all grown up as she takes her husband and son to an art gallery in West Hollywood.

The former X-Factor judge enjoys some fun in the sun with her two boys in Santa Barbara, Calif.

Family time starts early as the group heads to breakfast in West Hollywood.

Our resident Skinnygirl spends Valentine's Day with the love of her life. 

It's a busy weekend for the Hollywood couple as they run errands and take their son to a T-Ball game.

It's all pretty and pastel for the mom and her baby girl. 

The Aussie walks through Santa Monica, Calif. in full dad mode

Talk about multitasking! The actor backed his cute son while walking their dog on Runyon Canyon in L.A.

Tori introduced us to her miracle fourth baby.

So L.A. The 31-year-old Victoria's Secret Angel chats on her phone while strolling her son down the street. 

Xtina takes her son out shopping in Brentwood, Calif. 

Kendra isn't afraid of being that mom, embarassing her young son with affection. 

Not your typical soccer mom, Posh sits in the bleechers with her daughter as they watch her son's game. 

The actress teaches her daughter the cautions of crossing the street—especially in New York City streets. 

The lovebirds take her kids out to a fun-filled day at the Georgia Aquarium. 

We think it's safe to say this little guy has some of the coolest parents around. 

Charlie Sheen's exes plan a playdate with all of their kids together.

Mom on-the-go takes her daughter to gymnastics class in NYC. 

Ms. Duff has proven herself to be a wonderful mom to 11-month-old Luca. 

The actress carries all her essentials with her. 

The sisters each tote a kid to Sunday service. 

For the latest breaking news updates, click here to download the E! News App