Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Voice Message #3598

Merged
merged 69 commits into from
Jul 30, 2021
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
69350ef
Voice message UI initial implementation.
Jun 17, 2021
cb96886
Send voice message.
Jun 17, 2021
5676226
Voice message recording view implementations.
Jul 1, 2021
9d48b39
Voice message playback implementation.
Jul 1, 2021
7a1b138
Merge branch 'develop' into feature/ons/voice_message
Jul 6, 2021
6cb53c9
Fix visibility of microphone icon.
Jul 6, 2021
5856e56
Put voice message preference under labs.
Jul 7, 2021
8452f75
Lint fix.
Jul 7, 2021
b1c7cb3
Lint fixes.
Jul 7, 2021
a9beded
Lint fix.
Jul 7, 2021
3372177
Design review fixes.
Jul 7, 2021
b400f3c
Small cleanup
bmarty Jul 9, 2021
558cd6f
Cleanup
bmarty Jul 9, 2021
7a1c6b7
Move the style to the layout, for code clarity. Style are not reused,…
bmarty Jul 9, 2021
9dea519
Fix issue with play / pause button alignment
bmarty Jul 9, 2021
2792d73
Apply missing tint
bmarty Jul 9, 2021
a2671bc
Fix touchable area to delete the voice message
bmarty Jul 9, 2021
2c1335a
Theme for Toast
bmarty Jul 9, 2021
a69ac96
Fix bad timer display
bmarty Jul 9, 2021
79f4053
Use DimensionConverter
bmarty Jul 9, 2021
2ca0a99
Follow the spec
bmarty Jul 9, 2021
963eb9d
Fix missing duration in audio event, and move `AudioWaveformInfo` to …
bmarty Jul 9, 2021
984112e
Also copy waveform when sending again
bmarty Jul 9, 2021
db80ea6
Prefer testing attachment.waveform to see if it's a voice message
bmarty Jul 9, 2021
e391a13
Merge branch 'develop' into feature/ons/voice_message
bmarty Jul 9, 2021
0382ae8
Swipe to reply on voice message
bmarty Jul 9, 2021
c5b8755
Trick for a better alignment of voice message in the timeline.
bmarty Jul 9, 2021
e49290b
Fix mic visible for invite room mode.
bmarty Jul 12, 2021
8b852a2
Fix glitch when opening a room invite
bmarty Jul 12, 2021
9df874c
Merge branch 'develop' into feature/ons/voice_message
bmarty Jul 13, 2021
83bf48d
Fix denied permission issue
bmarty Jul 13, 2021
c69bc12
Be more precise if the timer is paused and resume.
bmarty Jul 13, 2021
78e9a4f
Use CountUpTimer
bmarty Jul 13, 2021
cae927b
Use CountUpTimer
bmarty Jul 13, 2021
fa7c1a7
Render the wave form more often
bmarty Jul 13, 2021
bf14fa9
Fix rendering issue of time when playing in the timeline - still buggy
bmarty Jul 13, 2021
7937c99
Untrack
bmarty Jul 13, 2021
bff2c6e
Better handling of Paused/Play state
bmarty Jul 13, 2021
6283846
Cleanup
bmarty Jul 13, 2021
a52d5f6
Renaming
bmarty Jul 13, 2021
4c5be39
Fix background color of Play/Pause button
bmarty Jul 13, 2021
14dbbee
Fix background color of voice message item
bmarty Jul 13, 2021
6530440
Fix an issue in the color
bmarty Jul 13, 2021
cf4e603
Fix background color of voice message recorder
bmarty Jul 13, 2021
9e0f3a1
Fix other color issue
bmarty Jul 13, 2021
f40e6b5
Fix other color issue
bmarty Jul 13, 2021
013174d
License file
bmarty Jul 13, 2021
95bb796
Fix color
bmarty Jul 13, 2021
6a0ea11
Follow the spec regarding waveform content
bmarty Jul 13, 2021
c938a30
Change filename.
bmarty Jul 13, 2021
df795d1
Cleanup
bmarty Jul 13, 2021
0cf10b2
Fix issue with waveform rendering
bmarty Jul 13, 2021
276808c
Fix issue in RTL
bmarty Jul 13, 2021
6ab9b46
Fix mic button color
bmarty Jul 15, 2021
bb742eb
Handle record/play error
bmarty Jul 15, 2021
6f947e9
Split to sub fun
bmarty Jul 15, 2021
bfc70be
Record voice on Android 21
bmarty Jul 15, 2021
343ea42
Fix issue on Android 21
bmarty Jul 15, 2021
13ae0ba
Convert voice message to be able to play on Android 28 and below
bmarty Jul 16, 2021
6da4f1d
Add comment
bmarty Jul 16, 2021
30bb918
Fix issue about move overflow.
bmarty Jul 16, 2021
6caa2b9
Fix issue with RTL
bmarty Jul 16, 2021
a119417
Code review fixes.
Jul 23, 2021
bd2ed4c
Stop playback when deleting record on locked mode.
Jul 23, 2021
4c8a8d8
Design review fixes.
Jul 27, 2021
cdd2fca
Design review fixes.
Jul 29, 2021
e945b89
Hide mic if there is a draft message.
Jul 30, 2021
cfe64ee
Update initial recording state to restore from background.
Jul 30, 2021
1aa706d
Lint fixes.
Jul 30, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ allprojects {
// Chat effects
includeGroupByRegex 'com\\.github\\.jetradarmobile'
includeGroupByRegex 'nl\\.dionsegijn'

// Voice RecordView
includeGroupByRegex 'com\\.github\\.Armen101'
}
}
maven { url 'https://oss.sonatype.org/content/repositories/snapshots/' }
Expand Down
16 changes: 16 additions & 0 deletions library/ui-styles/src/main/res/values/colors.xml
Original file line number Diff line number Diff line change
Expand Up @@ -128,4 +128,20 @@
<color name="vctr_chat_effect_snow_background_light">@color/black_alpha</color>
<color name="vctr_chat_effect_snow_background_dark">@android:color/transparent</color>

<attr name="vctr_voice_message_lock_background" format="color" />
<color name="vctr_voice_message_lock_background_light">#FFF3F8FD</color>
<color name="vctr_voice_message_lock_background_dark">#22252B</color>

<attr name="vctr_voice_message_playback_background" format="color" />
<color name="vctr_voice_message_playback_background_light">#FFE3E8F0</color>
<color name="vctr_voice_message_playback_background_dark">#FF394049</color>

<attr name="vctr_voice_message_play_pause_button_background" format="color" />
<color name="vctr_voice_message_play_pause_button_background_light">@android:color/white</color>
<color name="vctr_voice_message_play_pause_button_background_dark">#FF8E99A4</color>

<attr name="vctr_voice_message_recording_playback_background" format="color" />
<color name="vctr_voice_message_recording_playback_background_light">#FFE3E8F0</color>
<color name="vctr_voice_message_recording_playback_background_dark">#FF394049</color>

</resources>
154 changes: 154 additions & 0 deletions library/ui-styles/src/main/res/values/styles_voice_message.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>

<attr name="vctr_voice_message_lock_background_style" format="reference" />
<attr name="vctr_voice_message_playback_background_style" format="reference" />
<attr name="vctr_voice_message_play_pause_button_background_style" format="reference" />
<attr name="vctr_voice_message_recording_playback_background_style" format="reference" />

<style name="VoiceMessageLockBackground">
<item name="android:layout_width">52dp</item>
<item name="android:layout_height">160dp</item>
<item name="android:background">@drawable/bg_voice_message_lock</item>
<item name="android:backgroundTint">?vctr_voice_message_lock_background</item>
</style>

<style name="VoiceMessageMicButton">
<item name="android:layout_width">wrap_content</item>
<item name="android:layout_height">wrap_content</item>
<item name="android:background">?android:attr/selectableItemBackground</item>
<item name="android:layout_marginBottom">12dp</item>
<item name="android:layout_marginEnd">12dp</item>
<item name="android:src">@drawable/ic_voice_mic</item>
</style>

<style name="VoiceMessageSendButton">
<item name="android:layout_width">56dp</item>
<item name="android:layout_height">56dp</item>
<item name="android:background">@drawable/bg_send</item>
<item name="android:scaleType">center</item>
<item name="android:src">@drawable/ic_send</item>
</style>

<style name="VoiceMessageTimerIndicator">
<item name="android:layout_width">10dp</item>
<item name="android:layout_height">10dp</item>
<item name="android:layout_marginStart">20dp</item>
<item name="android:src">@drawable/circle</item>
<item name="tint">@color/palette_vermilion</item>
</style>

<style name="VoiceMessageTimer">
<item name="android:layout_width">wrap_content</item>
<item name="android:layout_height">wrap_content</item>
<item name="android:layout_marginStart">4dp</item>
<item name="android:lineSpacingExtra">4dp</item>
<item name="android:textSize">14sp</item>
<item name="android:textColor">@color/palette_gray_200</item>
</style>

<style name="VoiceMessageSlideToCancel">
<item name="android:layout_width">wrap_content</item>
<item name="android:layout_height">wrap_content</item>
<item name="android:gravity">center</item>
<item name="android:textColor">@color/palette_gray_300</item>
<item name="android:textSize">14sp</item>
<item name="drawableStartCompat">@drawable/ic_voice_slide_to_cancel_arrow</item>
</style>

<style name="VoiceMessageLockImage">
<item name="android:layout_width">wrap_content</item>
<item name="android:layout_height">wrap_content</item>
<item name="android:layout_marginTop">16dp</item>
<item name="android:src">@drawable/ic_voice_message_unlocked</item>
</style>

<style name="VoiceMessageLockArrow">
<item name="android:layout_width">wrap_content</item>
<item name="android:layout_height">wrap_content</item>
<item name="android:src">@drawable/ic_voice_lock_arrow</item>
</style>

<style name="VoiceMessageRecordingPlaybackLayout">
<item name="android:layout_width">0dp</item>
<item name="android:layout_height">0dp</item>
<item name="android:background">@drawable/bg_voice_playback</item>
<item name="android:layout_marginStart">16dp</item>
<item name="android:backgroundTint">?vctr_voice_message_recording_playback_background</item>
</style>

<style name="VoiceMessageDeletePlayback">
<item name="android:layout_width">wrap_content</item>
<item name="android:layout_height">wrap_content</item>
<item name="android:background">?android:attr/selectableItemBackground</item>
<item name="android:src">@drawable/ic_recycle_bin</item>
<item name="tint">@color/palette_gray_200</item>
</style>

<style name="VoiceMessagePlaybackTimerIndicator">
<item name="android:layout_width">10dp</item>
<item name="android:layout_height">10dp</item>
<item name="android:layout_marginStart">8dp</item>
<item name="android:src">@drawable/circle</item>
<item name="layout_goneMarginStart">24dp</item>
<item name="tint">@color/palette_vermilion</item>
</style>

<style name="VoicePlaybackControlButton">
<item name="android:layout_width">32dp</item>
<item name="android:layout_height">32dp</item>
<item name="android:background">@drawable/bg_voice_play_pause_button</item>
<item name="android:paddingStart">3dp</item>
<item name="android:paddingEnd">0dp</item>
<item name="android:layout_marginStart">8dp</item>
<item name="android:src">@drawable/ic_voice_play</item>
<item name="android:backgroundTint">?vctr_voice_message_play_pause_button_background</item>
</style>

<style name="VoicePlaybackTime" parent="VoiceMessageTimer">
<item name="android:layout_marginStart">6dp</item>
<item name="layout_goneMarginStart">24dp</item>
</style>

<style name="VoicePlaybackWaveform">
<item name="android:layout_width">0dp</item>
<item name="android:layout_height">0dp</item>
<item name="android:layout_marginStart">8dp</item>
<item name="android:layout_marginEnd">8dp</item>
<item name="chunkColor">@color/palette_gray_300</item>
<item name="chunkAlignTo">center</item>
<item name="chunkMinHeight">1dp</item>
<item name="chunkRoundedCorners">true</item>
<item name="chunkSoftTransition">true</item>
<item name="chunkSpace">2dp</item>
<item name="chunkWidth">2dp</item>
<item name="direction">leftToRight</item>
</style>

<style name="VoiceTimelinePlaybackLayout">
<item name="android:layout_width">0dp</item>
<item name="android:layout_height">wrap_content</item>
<item name="android:background">@drawable/bg_voice_playback</item>
<item name="android:minHeight">48dp</item>
<item name="android:paddingStart">8dp</item>
<item name="android:paddingTop">6dp</item>
<item name="android:paddingEnd">8dp</item>
<item name="android:paddingBottom">6dp</item>
<item name="android:backgroundTint">?vctr_voice_message_playback_background</item>
</style>

<style name="VoiceTimelinePlaybackControlButton">
<item name="android:layout_width">32dp</item>
<item name="android:layout_height">32dp</item>
<item name="android:background">@drawable/bg_voice_play_pause_button</item>
<item name="android:paddingStart">3dp</item>
<item name="android:paddingEnd">0dp</item>
<item name="android:src">@drawable/ic_voice_play</item>
<item name="android:backgroundTint">?vctr_voice_message_play_pause_button_background</item>
</style>

<style name="VoiceTimelinePlaybackTime" parent="VoiceMessageTimer">
<item name="android:layout_marginStart">6dp</item>
</style>

</resources>
5 changes: 5 additions & 0 deletions library/ui-styles/src/main/res/values/theme_dark.xml
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,11 @@

<item name="vctr_jump_to_unread_style">@style/Widget.Vector.JumpToUnread.Dark</item>

<!-- Voice Message -->
<item name="vctr_voice_message_lock_background">@color/vctr_voice_message_lock_background_dark</item>
<item name="vctr_voice_message_playback_background">@color/vctr_voice_message_playback_background_dark</item>
<item name="vctr_voice_message_play_pause_button_background">@color/vctr_voice_message_play_pause_button_background_dark</item>
<item name="vctr_voice_message_recording_playback_background">@color/vctr_voice_message_recording_playback_background_dark</item>
</style>

<style name="Theme.Vector.Dark" parent="Base.Theme.Vector.Dark" />
Expand Down
5 changes: 5 additions & 0 deletions library/ui-styles/src/main/res/values/theme_light.xml
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,11 @@

<item name="vctr_jump_to_unread_style">@style/Widget.Vector.JumpToUnread.Light</item>

<!-- Voice Message -->
<item name="vctr_voice_message_lock_background">@color/vctr_voice_message_lock_background_light</item>
<item name="vctr_voice_message_playback_background">@color/vctr_voice_message_playback_background_light</item>
<item name="vctr_voice_message_play_pause_button_background">@color/vctr_voice_message_play_pause_button_background_light</item>
<item name="vctr_voice_message_recording_playback_background">@color/vctr_voice_message_recording_playback_background_light</item>
</style>

<style name="Theme.Vector.Light" parent="Base.Theme.Vector.Light" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import io.reactivex.Single
import kotlinx.coroutines.rx2.rxCompletable
import kotlinx.coroutines.rx2.rxSingle
import org.matrix.android.sdk.api.query.QueryStringValue
import org.matrix.android.sdk.api.session.content.ContentAttachmentData
import org.matrix.android.sdk.api.session.events.model.Event
import org.matrix.android.sdk.api.session.identity.ThreePid
import org.matrix.android.sdk.api.session.room.Room
Expand Down Expand Up @@ -146,6 +147,10 @@ class RxRoom(private val room: Room) {
fun deleteAvatar(): Completable = rxCompletable {
room.deleteAvatar()
}

fun sendMedia(attachment: ContentAttachmentData, compressBeforeSending: Boolean, roomIds: Set<String>): Completable = rxCompletable {
room.sendMedia(attachment, compressBeforeSending, roomIds)
}
}

fun Room.rx(): RxRoom {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ data class ContentAttachmentData(
val name: String? = null,
val queryUri: Uri,
val mimeType: String?,
val type: Type
val type: Type,
val waveform: List<Int>? = null
) : Parcelable {

@JsonClass(generateAdapter = false)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import com.squareup.moshi.Json
import com.squareup.moshi.JsonClass
import org.matrix.android.sdk.api.session.events.model.Content
import org.matrix.android.sdk.api.session.room.model.relation.RelationDefaultContent
import org.matrix.android.sdk.internal.crypto.model.rest.AudioWaveformInfo
import org.matrix.android.sdk.internal.crypto.model.rest.EncryptedFileInfo

@JsonClass(generateAdapter = true)
Expand Down Expand Up @@ -50,7 +51,17 @@ data class MessageAudioContent(
/**
* Required if the file is encrypted. Information on the encrypted file, as specified in End-to-end encryption.
*/
@Json(name = "file") override val encryptedFileInfo: EncryptedFileInfo? = null
@Json(name = "file") override val encryptedFileInfo: EncryptedFileInfo? = null,

/**
* Encapsulates waveform and duration of the audio.
*/
@Json(name = "org.matrix.msc1767.audio") val audioWaveformInfo: AudioWaveformInfo? = null,

/**
* Indicates that is a voice message.
*/
@Json(name = "org.matrix.msc2516.voice") val voiceMessageIndicator: Any? = null
) : MessageWithAttachmentContent {

override val mimeType: String?
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
/*
* Copyright 2020 The Matrix.org Foundation C.I.C.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.matrix.android.sdk.internal.crypto.model.rest

import com.squareup.moshi.Json
import com.squareup.moshi.JsonClass

@JsonClass(generateAdapter = true)
data class AudioWaveformInfo(
@Json(name = "duration")
val duration: Int? = null,

@Json(name = "waveform")
val waveform: List<Int>? = null

)
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ import org.matrix.android.sdk.api.session.room.model.relation.ReplyToContent
import org.matrix.android.sdk.api.session.room.timeline.TimelineEvent
import org.matrix.android.sdk.api.session.room.timeline.getLastMessageContent
import org.matrix.android.sdk.api.session.room.timeline.isReply
import org.matrix.android.sdk.internal.crypto.model.rest.AudioWaveformInfo
import org.matrix.android.sdk.internal.di.UserId
import org.matrix.android.sdk.internal.session.content.ThumbnailExtractor
import org.matrix.android.sdk.internal.session.permalinks.PermalinkFactory
Expand Down Expand Up @@ -289,14 +290,20 @@ internal class LocalEchoEventFactory @Inject constructor(
}

private fun createAudioEvent(roomId: String, attachment: ContentAttachmentData): Event {
val isVoiceMessage = attachment.mimeType == "audio/ogg"
val content = MessageAudioContent(
msgType = MessageType.MSGTYPE_AUDIO,
body = attachment.name ?: "audio",
audioInfo = AudioInfo(
mimeType = attachment.getSafeMimeType()?.takeIf { it.isNotBlank() },
size = attachment.size
),
url = attachment.queryUri.toString()
url = attachment.queryUri.toString(),
audioWaveformInfo = if (!isVoiceMessage) null else AudioWaveformInfo(
duration = attachment.duration?.toInt(),
waveform = attachment.waveform
),
voiceMessageIndicator = if (!isVoiceMessage) null else Any()
)
return createMessageEvent(roomId, content)
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,6 @@ data class MultiPickerAudioType(
override val size: Long,
override val mimeType: String?,
override val contentUri: Uri,
val duration: Long
val duration: Long,
var waveform: List<Int>? = null
) : MultiPickerBaseType
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ internal fun Uri.toMultiPickerVideoType(context: Context): MultiPickerVideoType?
}
}

internal fun Uri.toMultiPickerAudioType(context: Context): MultiPickerAudioType? {
fun Uri.toMultiPickerAudioType(context: Context): MultiPickerAudioType? {
val projection = arrayOf(
MediaStore.Audio.Media.DISPLAY_NAME,
MediaStore.Audio.Media.SIZE
Expand Down
5 changes: 4 additions & 1 deletion vector/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,8 @@ android {

buildConfigField "im.vector.app.features.crypto.keysrequest.OutboundSessionKeySharingStrategy", "outboundSessionKeySharingStrategy", "im.vector.app.features.crypto.keysrequest.OutboundSessionKeySharingStrategy.WhenTyping"

buildConfigField "Long", "VOICE_MESSAGE_DURATION_LIMIT_MS", "120000L"

// If set, MSC3086 asserted identity messages sent on VoIP calls will cause the call to appear in the room corresponding to the asserted identity.
// This *must* only be set in trusted environments.
buildConfigField "Boolean", "handleCallAssertedIdentityEvents", "false"
Expand Down Expand Up @@ -339,7 +341,7 @@ dependencies {
implementation "androidx.recyclerview:recyclerview:1.2.1"
implementation 'androidx.appcompat:appcompat:1.3.0'
implementation "androidx.fragment:fragment-ktx:$fragment_version"
implementation 'androidx.constraintlayout:constraintlayout:2.0.4'
implementation 'androidx.constraintlayout:constraintlayout:2.1.0-beta02'
implementation "androidx.sharetarget:sharetarget:1.1.0"
implementation 'androidx.core:core-ktx:1.5.0'
implementation "androidx.media:media:1.3.1"
Expand Down Expand Up @@ -401,6 +403,7 @@ dependencies {
implementation "androidx.autofill:autofill:$autofill_version"
implementation 'jp.wasabeef:glide-transformations:4.3.0'
implementation 'com.github.vector-im:PFLockScreen-Android:1.0.0-beta12'
implementation 'com.github.Armen101:AudioRecordView:1.0.5'

// Custom Tab
implementation 'androidx.browser:browser:1.3.0'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,8 @@ fun MultiPickerAudioType.toContentAttachmentData(): ContentAttachmentData {
size = size,
name = displayName,
duration = duration,
queryUri = contentUri
queryUri = contentUri,
waveform = waveform
)
}

Expand Down
Loading