I use a couple of house rules in my DF game. My players know these from experience, although I haven't really spelled them out. I like using SM as a modifier to hit both in melee and missile combat, but penalizing big fighters for fighting human sized targets really goes against the spirit of DF. In the games and fiction I'm emulating and drawing inspiration from, being bigger makes you easier to hit with missiles but doesn't make it harder for you to hit human-sized heroes. Therefore:
"To hit" rolls are zeroed out to SM 0. You skills are rated for use scaled to a SM 0 human. Don't apply relative SM as a penalty to skill unless the target is smaller than SM 0. In that case, use relative SM.
In other words, positive SM never gives you a penalty to hit a smaller foe unless that foe is below SM 0. You're a SM+3 giant with Broadsword-16 fighting a SM 0 human? You roll against a 16. A SM+1 barbarian? Roll against a 17. A SM-2 halfling? 14. Conversely, a SM-2 halfling with Shortsword-15 rolls against an 18 to hit you, because your SM is +3. And yes, you roll against a 19 to fight another SM+3 giant. Since the game I play centers on SM-2 to +1 PCs, this isn't a huge problem. In any case, SM mods cap at +4 per the FAQ (specifically case 184.108.40.206).
We still use relative SM for small guys: a SM-2 halfling is -2 to hit for a human, but not for another halfling. A SM-1 goblin strikes a SM-2 halfling at a -1; a human strikes one at -2. A SM-10 ant strikes the halfling at +4, not that I ever roll for a single ant.
Why? This keeps giants and big dragons from needing incredible skill just to hit their human-sized foes. It also means they don't get so much greater defenses - giving, say, a SM+4 dragon skill 20 so it can hit a human on a 16 or less means it gets a parry of 13 with its claws, while just saying it has a 16 skill gives it a 20 to hit other dragons and a Parry of 11 allows for a normal attack/defense roll spread.
Relative SM does affect grappling in interesting ways in my games, but I'm using a house ruled set of Technical Grappling rules in my games that aren't ready for prime time reveal yet, but suffice it to say being bigger really helps you grapple people.
I won't vouch for this outside of DF, but it works in its intended setting quite well.