FEAT(client,images): Add animated image support to tooltips on the user view #6874
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The way images are handled for the `gif` file format when pasting and receiving one in the chat was here changed to support playing the animation. By default they do not play but whenever an animation is not running this is indicated by a play-icon, thereby differentiating them from still images on a glance. Whether the animation is paused is toggled by left-clicking it and reset by middle-clicking it. Saving a `gif` image file from log is also supported just as it is for other image file formats. Additionally, one can also toggle video controls for animations via the log context menu on them. This enables the following features: - Jumping to any point of the animation via the video bar - Viewing the current time and total time of the animation - Switching caching of all frames on or off - Switching loop mode between "Unchanged", "Loop" and "No loop" - Traversing frame-by-frame backwards or forward - Changing and resetting the playback speed Alternatively, animations can be controlled via keybindings as follows: - ENTER - Select the next animation - BACKSPACE - Select the previous animation - CTRL+0-9 - Select animation by the index entered - ESCAPE - Deselect animations - SPACE/K - Play/pause - Q - Reset playback - V - Show/hide video controls - C - Turn caching of all frames on/off - O - Switch to the previous loop mode - L - Switch to the next loop mode - , - Jump to the preceding frame - . - Jump to the next frame - N - Jump 5 frames backwards - M - Jump 5 frames forward - H - Jump 1 second backwards - J - Jump 1 second forward - F - Jump 5 seconds backwards - G - Jump 5 seconds forward - -/S - Decrease playback speed by 5% - +/D - Increase playback speed by 5% - W - Decrease playback speed by 25% - E - Increase playback speed by 25% - R - Reset playback speed To turn on caching most notably boosts performance when jumping to frames that are sequentially far apart due to `QMovie` only being able to switch frame in order from start to end and around when caching is off, though it usually plays fine regardless except for when playing in reverse in an animation with many frames. Reverse playback is also implemented here, so when decreasing the speed to less than zero it will play at that speed in reverse as expected, taking a speed-step that's twice as big if the speed would otherwise become zero so that the animation only pauses when it's not playing. As for loop mode, "Unchanged" is to use the in-built setting in the `gif` image file for how many times it is to repeat, whereas "Loop" and "No loop" override this behavior accordingly. The main limitation is the character limit on text messages for images. Currently this is usually set to 128 KB, which is very small for a `gif` image file and on top of this requires the data to be 33% smaller before being sent in base64 encoding. This limit should be at least ten times larger in order to fit many `gif` image files, or this should apply to another limit specifically for animations. Other than that, the functionality for pasting images from the clipboard itself, instead of by a file path from it, could not be implemented for `gif` image files due to not being able to get compatible formatted data from the MIME data received. Here are workarounds that can be applied to these limitations: - Configure the server to increase or disable the limit on image message length - Save copied `gif` images to file and then copy and paste those files Lastly, if settings were to be added to this feature, then here are some suggestions for those settings: - Play animations by default - Show video controls by default - Cache all frames by default - Specify the default loop mode - Specify the default playback speed
…s adjustments Made the following changes: General: - Move functions into more suitable classes, where `clearDocument` (renamed to `clear`) is placed in the `LogTextBrowser` class and `toggleVideoControls` is placed in the `AnimationTextObject` class - Add support for `width` and `height` attributes applied to animations - Add support for use with WEBP, MNG, (A)PNG and AVIF, which are other image formats that may contain animations. WEBP and MNG are included in Qt's additional library "Qt Image Formats", though only WEBP is bundled with official builds of Qt. CustomElements.cpp: - Reorganize video controls into a dedicated class, as well as related rather generic functions - Return default value after `switch` in `loopModeToString`, which would make compilers warn here if not all values of the enum are handled - Also clear the counter and focus index for custom items when the document is cleared - Select the last custom item if the number entered (CTRL+[series of 0-9] with the log in focus) is too high - Set the playback speed to the value it had at the previous reset if the speed is reset again while at the original speed - Add fullscreen mode, useable from the UI and via keybindings where F toggles it and Escape exits it - Add keybinding to invert the playback speed with X - Switch keybindings for jumping 5 frames backwards or forward to B and N as well as for jumping 5 seconds backwards or forward to U and I - Account for scrolling on both axes when getting the position of a click - Make video bar background brighter when caching is on to further distinguish it - Fix text object size so that it reliably changes the intrinsic size (layout) by applying the max size first (retroactive note: this was a side effect of not updating the layout manually for custom text objects which was properly fixed in the later commit "FIX(client,images): Leave scrolling position unchanged when exiting full screen") - Scale the animation to a min width for the video controls to ensure they fit - Use a regular still image for images that may contain an animation but do not have multiple frames MainWindow.cpp: - Use `enum class` instead of `enum` in new code, making it more type safe and specific in scope - Use function pointers instead of the `SLOT` macro for new actions in the log's context menu - Save the given image in another format if specified with the file extension for animations as well - Use precise image detection for context menu options via `formatAt` instead of `cursorAtPosition`, enabling direct detection of a text object at the given position, instead of looking for its detectable half which does not work when there are multiple such text objects in a row, and with more accuracy than just somewhere within its vertical space Log.cpp: - Reorganize the log routine for animations, where code having to do with creating and running an animation is placed in its own class, `AnimationTextObject`, while getting data and inserting animations is left to the `Log` class - Make the log routine for animations generic for all custom text objects, enabling more to be added alongside each other as well as regular HTML in the same message - Return early from `htmlWithCustomTextObjects` if the text to log is empty or too small to contain any tag - Use more stable specific reference points when parsing tags containing custom text object data - Still images use the given image format instead of JPG where possible, enabling transparency and maintaining the highest image quality (may still be reduced if the file size exceeds the given image message limit). The default image format is now PNG to enable transparency even if the format was not detected or supported, where PNG is in the base support for image formats in Qt. - Retroactively resolved merge conflict: Include upstream change for images to increase the max width from 600 to 1600 and max height from 400 to 1000
…orrectly When finding animations in a message where some of the images use image formats supporting animation, the search would incorrectly reuse the previous text object data and omit the invalid animation, breaking the loop's progress and missing the previous HTML along with the invalid animation. This change corrects it by only continuing the search for text object data at the start of the `while`-loop and by including the current `img`-tag if it does not contain a valid animation when inserting the previous HTML (custom text objects require that they are inserted separately in the document, hence this approach is used instead of something like replacing parts of a string and then setting the entire string).
…t objects to navigate When using keybindings to navigate custom text objects in the log, bound-checks were missing for moving to the next or previous item when there are no items. This change adds these bound-checks as well as universal use of `[]` instead of `at`.
…th animations Given that PNG images may contain an animation and optionally then use the file extension APNG, this change accepts either file extension. The list of file extensions for animations were also slightly rearranged to put WEBP next to AVIF.
… header Enables getting the file extension from a file header, i.e. the initial content of a file. This works when having an image in its original format, such as when downloading an image or decoding one from base64, but not when having a `QImage` due to it having read the data into its own format. File formats can then be treated differently even without a file path, which could be used to both set up custom objects for certain formats and to preserve the original format if that is supported in a resulting image after it is processed as a `QImage`.
Detection of SVG, ICO and CUR was added and that of several other image formats was corrected. File extensions are now consistently stored in lowercase instead in accordance with how Qt provides supported image formats.
…mated images The implementation of file extension association with animations was here changed from a static list to a dynamic one based on the read support in Qt for image formats supporting animation. If another such image format is added then the only part of the application that may require an update to support it is the function `isFileExt` in the class `Log` detecting the file extension based on the file header.
…d fallback to cover too large images in other formats Due to Mumble having a very limited image message length by default, the default image format is set to JPG again. The fallback for images now also includes images that could be written in the given format but are too large at any quality setting, trying those too in the default format if it did not work at all otherwise. Notably this means that transparency will be lost (filled with black) when copying images directly (without a file path) since these cannot have their original format made known when only exposed as a `QImage`. The quality may be reduced too but would likely be worse if it is even shown at all in another format with the same given image message length limit.
…ng an image The prompt for saving an image was here changed from having one filter with a few suggested image formats to multiple filters where there is one for each format with write support as well as one for the format of the data as-is if it is not already included. The filter for the current image's format is selected by default, which is made known when saving an animation and is otherwise JPG.
…ull screen When using keybindings to exit full screen the old position and size would be used, resulting in scrolling to near the top of the document instead of to where the text object interacted with is. Since you will have already scrolled to the text object's position in the document when entering full screen this may be ignored when exiting full screen. This change also tidies up some code regarding the text objects, including updating the layout when their size change.
By default custom text objects are ignored by Copy and Paste operations in Qt. This change adds support for text objects of animated images when copying selected content from the log. This change also tidies up some code, making a capture of `this` more specific.
The following parts of the video controls can now be scrolled: - The video bar, jumping 1 second forward or backwards - The playback speed buttons, adjusting the speed by 5% - The frame-by-frame buttons, switching to the next or previous frame - The loop mode, switching to the next or previous mode These now scrollable controls are in intentionally limited areas so as to avoid accidentally interfering with regular scrolling of the log. When in full screen the view may be scrolled as well to jump 1 second forward or backwards.
…evant variables by value
This change consistently uses the full name for functions that don't return or set a position or size. This is to improve readability.
This change replaces all occurrences of "Animation" regarding animated images in the name of enums, classes and functions with "ImageAnimation". This is to improve readability.
…andling more uniform for pasted images This change correctly includes the message for when the server prohibits images for translation. It also makes errors where a file cannot be read handled in a more uniform manner with the other image-pasting errors, only replacing the error cause in the error message.
…th `QTextDocument` This change removes the dependency on `LogTextBrowser` and enables adding support for animated images, as well as any other custom text objects if they were added in the future, using only the given document.
…text objects Since percent encoding is used on top of base64 encoding specifically for avatars for some reason, this change handles it by checking for and decoding the percent encoding first, thereby supporting both formats with only slightly more code.
…deo controls are shown This change makes each document have their own switch for showing video controls on animated images instead of one for the entire application. This makes the then inappropriate video controls never appear in tooltips and keeps the state separate between documents.
This change makes it clear and consistent when you have reached the end of an animated image by simply limiting jumps by frame to the beginning or end, just as when jumping by time.
…`nullptr` is not a valid input This is done to improve readability, making it more clear where passing `nullptr` works.
… switch in context menu This is done to accommodate the context menu of the rich text editor.
…om text objects on context menu as well
…nd use overload of `clear` correctly
…er view The UI was here changed to support animated images in tooltips on users and channels, enabling this to be viewed in comments and avatars when hovered.
GeneralUser01
force-pushed
the
gif-in-tooltips
branch
from
7fe3b68 to
d63abb5
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The UI was here changed to support animated images in tooltips on users and channels, enabling this to be viewed in comments and avatars when hovered.
Checks