When announcing the 15-man group, Cricket West Indies had said in a statement that Joseph "was unavailable for selection due to personal reasons".
The absence of the paceman is a big blow for the Windies after Joseph took six wickets in two Tests against Sri Lanka that helped towards a series victory.
"Yeah, it is a big loss. Mr Joseph has declined selection, Sammy told reporters. "That's the reality of the situation. He was selected. I mean, he's now coming from injury. And for reasons since, he has declined the selection.
"That is a decision that is way over my head. I leave that with the directors to come up with whatever solutions where contracted players get to decline selection."
The pace attack against Pakistan will now consist of Shamar Joseph, Kemar Roach and Jayden Seales with seam-allrounders Justin Greaves and Keemo Paul also in the squad.
Sammy added: "What I could tell you (is that) we have a bowling line-up that we've seen that could take 20 wickets. That's the first start to winning Test matches."
West Indies head coach Daren Sammy says he is proud of the way rookie spinner Vitel Lawes handled himself in his debut ODI series against New Zealand despite having never played a professional senior game previously.
The cricket world is mourning the death one of the game's most iconic players following the news that West Indies great Garfield Sobers has passed away at the age of 89.
West Indies captain Roston Chase hopes the Test series victory at home against Sri Lanka will be the "stepping stone" for better things to come in the longest format of the game.
Kemar Roach became just the fifth West Indies bowler to reach the milestone of 300 Test wickets as the Caribbean side thrashed Sri Lanka by an innings and 217 runs in the first Test in Antigua.
