• 0

2 Questions about Hash Strings


Question

When I generate a hash (say SHA256 or SHA512) , the hash string is composed of a combination of a-f and 0-9.

 

  1. Is there a way to generate a hash string that is composed of a-z, A-Z and 0-9? 
  2. Is there away to control what characters are used so if I only wanted m-z, A-L, 0-9 and "-_=*^#@!()[]{}<>;:,.?" that would be a possibility?
Link to comment
https://www.neowin.net/forum/topic/1369666-2-questions-about-hash-strings/
Share on other sites

11 answers to this question

Recommended Posts

  • 0

You could base36 encode the hash output to give you a string composed of a-z0-9 (or write a simple custom cipher to map to whatever set of characters you want) but I can't think of a reason why you would want to do this?

  • 0

Understanding why it's needed is my business. But thank you.

 

Secondly ,PHP does this when creating a session. You are able to customize it's sid_bits_per_character to 6, which does a-zA-Z and 0-9; thus I assumed there is a method to specify what values you want to be included as the components of the hash.

  • 0

You assumed wrong. What PHP is doing is completely independent of the hashing method, it's simply taking the bits returned from (any) hashing method and rather than displaying them as a hexadecimal representation it's encoding them into a string using a character set of their choosing, just as I said you could do:

 


static char hexconvtab[] = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ,-";

static void bin_to_readable(unsigned char *in, size_t inlen, char *out, size_t outlen, char nbits) /* {{{ */
{
	unsigned char *p, *q;
	unsigned short w;
	int mask;
	int have;

	p = (unsigned char *)in;
	q = (unsigned char *)in + inlen;

	w = 0;
	have = 0;
	mask = (1 << nbits) - 1;

	while (outlen--) {
		if (have < nbits) {
			if (p < q) {
				w |= *p++ << have;
				have += 8;
			} else {
				/* Should never happen. Input must be large enough. */
				ZEND_ASSERT(0);
				break;
			}
		}

		/* consume nbits */
		*out++ = hexconvtab[w & mask];
		w >>= nbits;
		have -= nbits;
	}

	*out = '\0';
}

https://github.com/php/php-src/blob/master/ext/session/session.c#L269

 

 

I asked why because I hope you're not using this for security purposes, based on the fact that you had to ask this question in the first place you're more likely to end up reducing security rather than increasing it. 

Edited by ZakO
  • Like 1
  • Thanks 1
  • 0

1. No you can only have one of A-Z or a-z.  This can be done by encoding the string that your hashing algorithm to something that is hex.

2. No, this defeats the point of a hash.  I can't think of a good reason for doing this.

  • 0
  On 05/08/2018 at 11:20, Fahim S. said:

1. No you can only have one of A-Z or a-z.  This can be done by encoding the string that your hashing algorithm to something that is hex.

2. No, this defeats the point of a hash.  I can't think of a good reason for doing this.

Expand  

Thanks.

It's funny when I make inquiries, rather than answering I am offered personal opinions of understanding.  

 

While you offered some answers, you added your ego (or lack of worldly experience) in to the mix.  You do not need to know why I want something.  The comment of "I can't think of a good reason for doing this" is naive and immature.  Of course you cannot think of a good reason to do this; it's because you haven't lived my life; surely you understand that. But mostly that comment is completely a relevant.

 

In future, just answer the question and don't interject your immaturity in to your response.

 

Cheers mate.

Edited by Brian Miller
  • 0
  On 05/08/2018 at 10:51, ZakO said:

You assumed wrong. What PHP is doing is completely independent of the hashing method, it's simply taking the bits returned from (any) hashing method and rather than displaying them as a hexadecimal representation it's encoding them into a string using a character set of their choosing, just as I said you could do:

  


static char hexconvtab[] = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ,-";

static void bin_to_readable(unsigned char *in, size_t inlen, char *out, size_t outlen, char nbits) /* {{{ */
{
	unsigned char *p, *q;
	unsigned short w;
	int mask;
	int have;

	p = (unsigned char *)in;
	q = (unsigned char *)in + inlen;

	w = 0;
	have = 0;
	mask = (1 << nbits) - 1;

	while (outlen--) {
		if (have < nbits) {
			if (p < q) {
				w |= *p++ << have;
				have += 8;
			} else {
				/* Should never happen. Input must be large enough. */
				ZEND_ASSERT(0);
				break;
			}
		}

		/* consume nbits */
		*out++ = hexconvtab[w & mask];
		w >>= nbits;
		have -= nbits;
	}

	*out = '\0';
}

https://github.com/php/php-src/blob/master/ext/session/session.c#L269

 

 

I asked why because I hope you're not using this for security purposes, based on the fact that you had to ask this question in the first place you're more likely to end up reducing security rather than increasing it. 

Expand  

 

Thanks dude, that's what I thought too.  I like your idea of the Base encoding it, I may use Base56.

 

The reason I had asked is because I wanted to learn about forming such strings, and not necessarily for any foolish attempt at security.  The formation of BitCoin addresses such as "1BoNtSLRHtKNngkdx3e0bR7gb53L3TtpYt" first peeked my curiosity, then when I discovered PHP sessions can also include  a-zA-Z and 0-9 when setting it's sid_bits_per_character to 6 prompted me to enquire with learned people here.

 

  • 0
  On 06/08/2018 at 03:26, Brian Miller said:

Thanks.

It's funny when I make inquiries, rather than answering I am offered personal opinions of understanding.  

 

While you offered some answers, you added your ego (or lack of worldly experience) in to the mix.  You do not need to know why I want something.  The comment of "I can't think of a good reason for doing this" is naive and immature.  Of course you cannot think of a good reason to do this; it's because you haven't lived my life; surely you understand that. But mostly that comment is completely a relevant.

 

In future, just answer the question and don't interject your immaturity in to your response.

 

Cheers mate.

Expand  

Err... ok. 

 

As we are offering tips to one-another let me give you one: a bit of context can go a long way in getting an answer.  Developers like solving problems, and without detail of the underlying problem it is difficult to help. 

 

The comment was intended to probe for context (suggest you read about the 5 whys) so that I can try my best (within the bounds of my knowledge) to help you come to an answer quicker.  I apologise for the negative impression that you drew from it.

  • Like 2
  • 0
  On 06/08/2018 at 03:26, Brian Miller said:

Thanks.

It's funny when I make inquiries, rather than answering I am offered personal opinions of understanding.  

 

While you offered some answers, you added your ego (or lack of worldly experience) in to the mix.  You do not need to know why I want something.  The comment of "I can't think of a good reason for doing this" is naive and immature.  Of course you cannot think of a good reason to do this; it's because you haven't lived my life; surely you understand that. But mostly that comment is completely a relevant.

 

In future, just answer the question and don't interject your immaturity in to your response.

 

Cheers mate.

Expand  

 

Mate, I see your post count and reputation, but this doesn't mean you should act like an a**hole  You are not paying those people to have such expectations for their answers. From my point of view those were relevant and polite answers and doing their best to help you.

 

...and yes adding my ego is perfectly fine on public forum.

 

Have a nice day!

  • 0

Hello,

 

SHA-256 and SHA-512 output their results in hexadecimal notation, which is why you see 0-9 and a-f used in the results--those are the sixteen digits which compose hexadecimal notation.

 

Instead of having to re-write the hashing algorithms to provide your own numbering system, perhaps it would be better to use something like SSDeep, instead, which supports a larger encoding set?

 

Regards,

 

Aryeh Goretsky

 

  • 0
  On 06/08/2018 at 03:26, Brian Miller said:

You do not need to know why I want something.

Expand  

You clearly do not understand how forums work... That you got the answers you got is way more than I would ever in a million years given you..  With such a comment when asked why..

  • 0

Just encode the hash, shortest practical encoding I can find is base85.

 

Encoding: input -> SHA256/512 -> base85

Decoding: base85 -> SHA256/512 -> Find input data with hash

 

There are multiple common base encodings: base2(1), base10(2), base16(3), base32, base36, base58, base64(4), base85, base91(5), base128(6)

 

Above base encodings have a default character set of X characters that are being used for encoding, but it's possible to replace those with your own character set.

 

  1. Base encoding of binary data (0 & 1)
  2. Base encoding of a decimal number (0-9)
  3. Base encoding of hexidecimal string like a SHA512 hash (A-F0-9)
  4. Base encoding commonly used for encoding binary data to a string to embed it in websites
  5. Base encoding with most printable characters
  6. Base encoding of a byte and ascii string

 

BUT

If you actually try to encode your hash you will find the string doesn't become shorter ?

 

Original:

seahorsepip

SHA512: 

F9AA2F6D639C026E3325F31247E8253987D6EC6EEC7E93764F9F3CC25D08FABA7DF95FAF94779CACF22D72F96EEE88D46C90A8CE727944218A1DC272EDA29084

base85: 

mMA+2gdBe&hzWuXfFUKPgCZ^zmL=+Og=E&.gbQ<Fi5:k-mme$4mmf15iwSPLg!6F{gEBI+hafbTmNovbh:*a}hax(3iw-VNiyu81mLV.!h.)&.hBQ56i5<q.hBxFUk@.k2h.)rLg=c%xi6/n!lOZOQmmoA%iwrAH

 

Why doesn't it become shorter?

When you create a hash from data it returns a hash string in the hexadecimal format, also known as base16.

So when you encode the hash as a string using base85 you actually tell the base85 encoder that the input is an ascii string (base128), so that means you're encoding a base128 string to a base85 string which results in a longer instead of shorter string!

 

How to fix this?

Make sure to actually let the base85 encoder know that the input format is base16.

So to do that you can actually convert the hex string to bytes(base128) with hex2bin in php for example and use those bytes as input for the base85 encoder.

This means the encoding would be: input -> SHA256/512(base16) -> bytes(base128) -> base85

 

Original:

seahorsepip

SHA512: 

F9AA2F6D639C026E3325F31247E8253987D6EC6EEC7E93764F9F3CC25D08FABA7DF95FAF94779CACF22D72F96EEE88D46C90A8CE727944218A1DC272EDA29084

base85: 

}kM8+w1iQrgBqS9n9zlVHT$*f)0+dwpOf+^t)QqrEFElNLY@U0[?1[TzTJtgy(>QvA^p4@IxfMO)v]X}

And even shorter base91 (only 1 char shorter in this example):

?e#a_*!Og$d0Rh"Y4Qx.=8}^zpmb~^B4aGWI;`W=?}5&b`B3w0Exl`S[GYF#fG9.1,vcLH]LR%LxhzQ

 

And the encoded string is now shorter :D

 

Php libraries to do this:

hex2bin: http://php.net/hex2bin

base85: https://github.com/tuupola/base85

 

So php code to create a shorter hash:

$shorterHash = $base85->encode(hex2bin(hash($file)))

 

Update:

Seems like you wanted to create a custom base56 encoding, to do that we could manually create functions encode and decode it:

$base56_digits = '0123456789ABCDEFGHIJKLMNOPQRSTVWXYZabcdefghijklmnopqrstv';
$custom_digits = 'mnopqrstvwxyzABCDEFGHIJKL0123456789-_=*^#@!()[]{}<>;:,.?';

function encode($base16) {
    global $base56_digits, $custom_digits;

    $base56 = base_convert($base16, 16, 56);
    $custom = strtr($base56, $base56_digits, $custom_digits);

    return $custom;
}

function decode($custom) {
    global $base56_digits, $custom_digits;

    $base56 = strtr($custom, $custom_digits, $base56_digits);
    $base16 = base_convert($base56, 56, 16);

    return $base16;
}

But above doesn't work since php base_convert is limited to base36 :(

Instead you can use a magnificent 3rd party library: https://github.com/ArtBIT/base_convert

 

And then you have:

$custom_digits = 'mnopqrstvwxyzABCDEFGHIJKL0123456789-_=*^#@!()[]{}<>;:,.?';

function encode($base16) {
    global $custom_digits;

    return math\base_convert($base16, 16, $custom_digits);
}

function decode($custom) {
    global $custom_digits;

    return math\base_convert($custom, $custom_digits, 16);
}

Original:

seahorsepip

SHA512: 

F9AA2F6D639C026E3325F31247E8253987D6EC6EEC7E93764F9F3CC25D08FABA7DF95FAF94779CACF22D72F96EEE88D46C90A8CE727944218A1DC272EDA29084

Custom base56: 

n<^(q8}=_G@x0;B1]K6zD-DF*96yE-6L#_>K8vJ},vCz02m,8yB][4qA^12>.pw>2-?_m,{0L<qFCK:K,2@04)3s:

 

TL;DR

All data is encoded in a specific base, data can be represented as a shorter string by increasing it's base and can be respresented with a smaller character dictionary by decreasing it's base.

 

Oftopic:

  Quote

Stop the bickering back and forth, we're here to learn things and help each other, if someone doesn't want to share why he wants to do something then that's his right.

Though that doesn't mean that you have to be rude about it, if you don't want to share the why, let others know in a respectful manner.

Expand  

 

  • Like 2
This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • What's wrong with a CPU from 2021? I'm still using laptops with CPU's that came out 2018-2020 ... they run great.
    • It’s encouraging that Intel is still competing, the best they can, in the gpu space.
    • Windows 11 KB5062233/ KB5060843/ KB5062197/ KB5061090 setup, recovery updates released by Sayan Sen This week Microsoft released non-security preview updates for Windows 11 22H2 and 23H2 under KB5060826 as well as for 24H2 under KB5060829. The company also published dynamic updates alongside them. Dynamic updates bring improvements to the Windows Recovery in the form of Windows Recovery Environment (WinRE) updates, also called Safe OS updates, as well as to the Setup binaries in the form of Setup updates. These Dynamic Update packages are meant to be applied to existing Windows images prior to their deployment. These packages include fixes to Setup.exe binaries, SafeOS updates for Windows Recovery Environment, and more. Dynamic Updates also help preserve Language Pack (LP) and Features on Demand (FODs) content during the upgrade process. VBScript, for example, is currently an FOD on Windows 11 24H2. Both setup and recovery updates were released. The changelogs are given below. First up we have the Setup updates: Up next, we have the recovery updates: Microsoft notes that the Recovery updates will be downloaded and installed automatically via the Windows Update channel. The Setup updates, however, need to be downloaded and installed manually. You can avail them on Microsoft's Update Catalog website: KB5062233, KB5060843, KB5062197, and KB5061090.
    • Gaming mouse review: Keychron Lemokey G2 8K Wireless, high performance, light in weight by Robbie Khan Available in the UK and USA, the G2 is a wireless gaming mouse by Lemokey, the customised gaming peripheral arm of Keychron, a brand many readers will know well. I've checked out Keychron keyboards recently, models such as the top-tier Q6 Max, and the bargain priced B6 Pro. I never got the chance to check out the G1, but the G2 seems to put itself firmly in-between competing 8K gaming mice out there whilst coming in at a decent sum under £100/$100. The other benefit is that it uses Keychron Launcher, the web-based software that has been a staple part of Keychron keyboards for a long time now. I rate the launcher highly and have rarely had a problem with it, so it is great to see a robust piece of software being used on an affordable gaming mouse that's not only built well, but performs well, too. As it's a high performance gaming mouse with a flagship PixArt sensor, I will be comparing the G2 directly against my personal mouse, the similarly sized Pulsar Feinmann F01. Both mice are priced wildly differently, so it was interesting to compared several factors between them to see if one bests the other where it matters most, or if it just comes down to individual preferences. Let's check out the specs... Keychron Lemokey G2 SKU G2-A1 Sensor PixArt 3950 IPS 750 MCU Realtek 8762G Connectivity Bluetooth 5.1 / 2.4 GHz / Wired (type-C cable) Polling Rate Up to 8000 Hz (2.4 GHz / wired mode), 125 Hz (Bluetooth mode) Motion Sync Yes Angle Snapping Yes Acceleration 50g Cable Detachable Type-C to C Cable + Type-A to Type-C adapter Dongle Micro USB-A dongle Main switches Huano Micro Switch Switch lifespan 80 Million Clicks Battery 300 mAh Battery life 60 Hours (Under 1000Hz) 35 Hours (1000-4000Hz) 20 Hours (4000-8000Hz) Skates Teflon / PTFE Lift Off Distance 0.7 / 1.0 / 2.0 mm Material (Body and Grip) ABS Dimensions 118mm / 62.6mm / 38.2mm Tracks on glass (min. 4 mm thickness) Yes (max 1000Hz report rate) Weight 52 ± 3 g Colours Black / White Price £73.99 / $69.99 Fit and finish Made of all ABS, it certainly doesn't feel anywhere near as nice as the F01, but then again it's priced much lower, so no complaints there. To me it feels more like a previous generation Endgame/Zowie mouse, before they started to use the more grippy textured finish. It's a fine finish but how long that lasts only time will tell. With the Endgame mice I always found that the side buttons would go shiny first, and I suspect the exact same to apply here with the G2, since the size and shape of them is similar, as is the material quality. Speaking of side buttons, my personal preference is flat and large buttons, the long and pointed ones like those on the G2 shown above and GAMIAC PX71 at the bottom always felt a bit awkward to me for naturally resting a thumb on, whereas the large surface area of the ones found on the Feinmann and others like it feel more ergonomic and comfortable for long usage sessions. Pulsar Feinmann F01 (ergo shape) Lemokey G2 (ambidextrous shape) Under my 19cm hand, the G2 feels comfortable and stable for both claw and palm grip styles, though my usual style is a hybrid approach. I like to pinch a mouse with a thumb and ring finger, then articulate forward or backward movement with them to adapt to different play-styles. Both mice have a sloping back hump that works brilliantly for this and i had no problem getting comfortable with the G2 here. The underside of the G2 hides a cool feature I wish more more mice makers adopted, can you spot what it is in the photo below? That little flap at the bottom to store the USB dongle, it's convenient, especially for gamers who travel with their mice or laptop gamers. The skates applied are PTFE and cover three zones on the underside, sadly I forgot to take a photo of them before swapping them out to my preferred skates, the Wallhack Pro UWMP yellow dots. I found no problem with glide resistance on the stock skates, though find dots glide nicer overall and these yellow dots slide around like butter. Lemokey states 52g for the weight, give or take 3g, so my scales measure up perfectly here. It's not as lightweight as the Feinmann which currently is 45g, but it's still considered super-light and honestly, under the hand, this weight difference on dot skates is hard to tell apart. Gripping the G2 hard on both sides shows no obvious issues with flexing, so the internal construction is also up to scratch and exactly what i would expect in this price range. The differences start to be noticed when you are using the G2 as a combination mouse, both in gaming and desktop use. The wheel is thinner than the one on the Feinmann and other gaming mice with fat wheels. I much prefer a nice wide wheel, the diameter is also much larger as can be seen with the overhead photo above. The button tops also have a distinct difference in feel when actuating the switches. there is more of a hollow feel to the G2's tops, as well as slightly more travel post-click. the Huano switches feel and sound excellent, though, but I think the ABS contact surface on the outside could have been slightly more distinct in feel. Here is a demo of how the main switches sound, you can also hear the switch tops clap if you pay attention: Likewise the side buttons have extra play after the switches actuate, something the Feinmann and others in the higher price category don't tend to have. Features & software Aside from the usual features that all gaming mice support these days, such as Motion Sync, lift off distance and Macro, the big feature with the G2 is that is uses Keychron Launcher, the web-browser based software. All changes are stored directly onto the on-board memory, here is what the sections within it look like: I found no bugs with the implementation here, and unlike other Keychron wireless devices I have used with Launcher in the past, the G2 connects and can be customised in it over both wired and wireless. Just be aware that the firmware update option when connected via wireless will only check the dongle's firmware. To check the G2's firmware, a USB cable will need to be connected. Performance Whether on the desktop in Windows or in games, the cursor and motion performance is excellent, though this is to be expected from all modern gaming mice, regardless of price. The higher priced mice tend to have better use of high quality materials, and implementation of software and physical features. I'm currently pacing through nine games, all mixed genre, and in each of them I had no troubles quickly getting comfortable with the G2. If you're used to an Endgame XM2we sized mouse, then this will be just as familiar to you. The convenience of the DPI button at the top of the mouse instead of underside makes instant switching simpler, although the button can be remapped to do something else for those who don't care about DPI toggling. Here is a demo of the G2 playing Doom: The Dark Ages: The sensor tracking performance at 8K was perfect, aside from the few moments my physical movement on the mousepad slowed down causing some drops in the Razer mouse tracking measurement graph below: A consistent 7900-8000Hz polling was observed, though keep in mind that using such high polling rates will impact CPU performance whilst gaming. How much this affects framerates will boil down to the CPU you have. On my i7 12700KF there was no change whilst gaming, but during the measurement above I did observe 13% CPU package utilisation. In practice, though, I saw no performance difference playing at 8K versus 2K or even 1K. Battery life will drastically be impacted at 8K polling rates for obvious reasons, and whilst many prefer 4K, there is a growing trend to just stick to 2K which feels the safe middle ground of great battery performance, whilst still being suitably responsive on paper for even the most fast-paced of gaming sessions. Conclusion The Lemokey G2 has proven to be a rather excellent mouse, not just for gaming, but general use, too. Though being excellent doesn't give it any special status, as there are equally excellent mice out there that cost the same, less and even more. A buying decision will come down to individual needs, do you value the use of a browser-based software tool like Keychron Launcher? If so, then this is right up your street. Do you want something lightweight but also supports the gaming features and feel in the hand as more expensive mice? Maybe the G2 will satisfy. It's not totally perfect, nothing ever is, the thumb buttons could have been a bit wider, and the switch caps have more travel after clicking than what I am comfortable with, but otherwise this is a great mouse with features that rival the competition. The sensors on gaming mice these days isn't a key selling factor any more either, since even entry level gaming mice are capable of precision tracking and speed that was only possible on the top-tier models of the past, like many things in tech now, the point of diminishing returns has been reached, and brands have to work harder at giving us consumers a unique selling point to attract interest. I believe the G2 has at least one USP (Keychron Launcher), it also helps that it's a very comfortable mouse to use for all-day sessions.
    • Delays can happen any time and names changed, but the joke was still there to be made.
  • Recent Achievements

    • Week One Done
      Alexander 001 earned a badge
      Week One Done
    • Week One Done
      icecreamconesleeves earned a badge
      Week One Done
    • One Year In
      PAC0 earned a badge
      One Year In
    • One Month Later
      PAC0 earned a badge
      One Month Later
    • One Year In
      Shahmir Shoaib earned a badge
      One Year In
  • Popular Contributors

    1. 1
      +primortal
      564
    2. 2
      +FloatingFatMan
      189
    3. 3
      ATLien_0
      185
    4. 4
      Skyfrog
      113
    5. 5
      Xenon
      109
  • Tell a friend

    Love Neowin? Tell a friend!